Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RAM requirements? #17

Open
swartchris8 opened this issue Oct 20, 2017 · 5 comments
Open

RAM requirements? #17

swartchris8 opened this issue Oct 20, 2017 · 5 comments
Assignees
@sandroacoelho
Labels
enhancement question

Comments

@swartchris8
Copy link

Running:

docker run -i -p 2222:80 dbpedia/spotlight-english spotlight.sh

Resulted in the process being killed, could this be due to running out of memory? I can see in spotlight.sh you require 20GB for the java heapsize so could this cause the below behaviour? What is the minimum RAM requirement to run dbpedia spotlight from docker?

Logs:

$ docker run -i -p 2222:80 dbpedia/spotlight-english spotlight.sh
Unable to find image 'dbpedia/spotlight-english:latest' locally
latest: Pulling from dbpedia/spotlight-english
88286f41530e: Pull complete 
720349d0916a: Pull complete 
9431a0557160: Pull complete 
89c0205cd042: Pull complete 
8be3620205bc: Pull complete 
f24c3713766d: Pull complete 
Digest: sha256:ca9577202bf7d490bde06288339df98ffcabb12da7c3b4841ac3385924d1c67d
Status: Downloaded newer image for dbpedia/spotlight-english:latest
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (43 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1300 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (23969 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (42725 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
Killed
@sandroacoelho sandroacoelho self-assigned this Oct 20, 2017
@sandroacoelho
Copy link
Collaborator

sandroacoelho commented Oct 20, 2017 •
edited

Hi @swartchris8 ,

Your question reminds me that we must make a benchmark test to adjust our memory parameters at Docker.

Anyway... It seems that you do not have enough RAM to run it. You must have at least 8GB RAM free to run English Spotlight. If you have it, please tell us what version Docker version are you using and your S.O.

All the best,

@swartchris8
Copy link
Author

Hey @sandroacoelho ,
Thanks for your quick reply I am on a Mac, docker version 17.09.0-ce-mac35 (19611) and I only have about 5GB of RAM after on an 8 GB RAM machine with docker running.
Cheers,

@owensacco
Copy link

Hi @sandroacoelho

I am having a similar issue on a Win 10 machine using docker version 17.09.0-ce-win33 (13620) - see logs below. Note: I am running this on 32GB of RAM.

Thanks!

logs:
docker run -i -p 2222:80 dbpedia/spotlight-english:latest spotlight.sh
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (43 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1342 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (23352 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (29628 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
Killed

@sandroacoelho
Copy link
Collaborator

sandroacoelho commented Oct 27, 2017 •
edited

Hi @owensacco,

Under linux we know that it works... I will test it under Windows :(

:/home/spotlight# docker run -i -p 2222:80 dbpedia/spotlight-english spotlight.sh
Unable to find image 'dbpedia/spotlight-english:latest' locally
latest: Pulling from dbpedia/spotlight-english
88286f41530e: Already exists 
720349d0916a: Already exists 
9431a0557160: Already exists 
4d91280b983d: Pull complete 
2852514d6a0b: Pull complete 
51a808189d57: Pull complete 
Digest: sha256:0798c91bafbd5d8e5d5487481a1b322a2173c613c112d34bc103e672ad478cd1
Status: Downloaded newer image for dbpedia/spotlight-english:latest
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (33 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (428 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (11007 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (2735 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1430 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryContextStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (28681 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading FSADictionary...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (373 ms)
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 1 disambiguators.
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 2 spotters.
Oct 27, 2017 12:34:21 AM com.sun.grizzly.Controller logVersion
INFO: GRIZZLY0001: Starting Grizzly Framework 1.9.48 - 10/27/17 12:34 AM
Server started in / listening on http://0.0.0.0:80/rest
[main] INFO org.dbpedia.spotlight.web.rest.Server -  Core threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server -   Max threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server - Max POST size: 2097152
Oct 27, 2017 12:34:21 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
 org.dbpedia.spotlight.web.rest.resources
Oct 27, 2017 12:34:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
 class org.dbpedia.spotlight.web.rest.resources.Feedback
 class org.dbpedia.spotlight.web.rest.resources.Candidates
 class org.dbpedia.spotlight.web.rest.resources.Annotate
 class org.dbpedia.spotlight.web.rest.resources.Disambiguate
 class org.dbpedia.spotlight.web.rest.resources.Spot
Oct 27, 2017 12:34:25 AM com.sun.jersey.api.core.ScanningResourceConfig init
INFO: No provider classes found.
Oct 27, 2017 12:34:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM

@collarad
Copy link

FYI: In my case, I increased the Java heap size (Xms/Xmx) in my docker container, as described in the Wiki->Troubleshooting (https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki/Troubleshooting), and it works now.

docker run -i -e JAVA_OPTS='-Xmx16g' -p 2222:80 dbpedia/spotlight-english spotlight.sh

@dpicca dpicca mentioned this issue Mar 17, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sandroacoelho sandroacoelho

Labels
enhancement question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@sandroacoelho @swartchris8 @collarad @owensacco