No DocIDs will be created if maxPagesToFetch is reached (most times).

[dgoiko]

Fixes #413

This is not a thread safe solution. One thread may fill the DocIDServer while another thread is looping, however, the amount of memory wasted will be decreased.

The current problem with this hotfix:
Supose there's only 1 slot left to fill maxPagesToFetch

1. Thread A creates a new docID (docA). DocIDServer will see that there's room for pages, so it will create it.
2. Thread B creates a new docID (docB)
3. Thread C schedules docA. It is accepted.
4. Thread D schedules docB. It is rejected, but it has been already created.

There should be a method to clean docB from DocIDServer in when it gets rejected, making the Frontier aware od DocIDServer, but since CrawlController has setFrontier and setDocIdServer methods, this wouldn't be safe either, as someone playing around with multiple DocIDServers and Frontiers may cause unpredictable situacions.

Since it isn't a perfect solution either, I decided to keep the hotfix as simple as possible. However, if you feel like it would be OK, I can quickly add another commit that allows the Frontier to remove unscheduled URLs from DocIDServer, given the problem stated above.