-

 

Three Months of Crawling. - 25/01/2015 Public

Yioop has been crawling for over three months now and is somewhere between 300 and 400 million pages. The first month was a little bit slow as each of my Mac Minis was updated to OSX Mavericks and I was still working out the kinks in the latest iteration of the crawler. I have now also switched from the slightly over a year old crawl that I had before as a default index to the current active crawl. This will probably make search results a little slow for now because everything is being served off regular hard drives. I am hoping to have saved for large enough SSDs to put the dictionary and posting lists on by the time I stop the current crawl in a few months.

All of the information learned working out the kinks in the current crawl was used in creating Version 2.0 of Yioop which was just released today! Here is the Official Yioop Version 2.0 Release Announcement.

Last Edited: 25/01/2015
Yioop has been crawling for over three months now and is somewhere between 300 and 400 million pages. The first month was a little bit slow as each of my Mac Minis was updated to OSX Mavericks and I was still working out the kinks in the latest iteration of the crawler. I have now also switched from the slightly over a year old crawl that I had before as a default index to the current active crawl. This will probably make search results a little slow for now because everything is being served off regular hard drives. I am hoping to have saved for large enough SSDs to put the dictionary and posting lists on by the time I stop the current crawl in a few months. <br /> All of the information learned working out the kinks in the current crawl was used in creating [[https://seekquarry.com/?c=static&p=Downloads|Version 2.0 of Yioop]] which was just released today! Here is the [[https://yioop.com/?c=group&a=groupFeeds&just_thread=669|Official Yioop Version 2.0 Release Announcement]].
 
[X ]