[Earthquake] Switch datasource

[Burgje]

Reading data from : http://apis.is/earthquake/is keeps giving the data with as most recent date 13-10-2017

This is the code in python which was working before:

import requests
import json

data_json = requests.get("http://apis.is/earthquake/is").json()
print (json.dumps(data_json, indent=4))

[koddsson]

Most likely related to #399

[kristjanmik]

@Burgje I see that the datasource is showing older data now. Does this error still persist?

[Burgje]

@kristjanmik Yes it still is the case. I have just rerun the script and this the latest set;
date = 2017-10-13
time = 12:07:24
latitude = 63.976
longitude = -21.949
depth = 1.1
size = 0.6
quality = 58.73

[koddsson]

@kristjanmik Are you seeing newer data? The latest data I see in the data source is consistent with the results in apis.is for me.

[Burgje]

just now 19-10-2017 9:02 via API:
date = 2017-10-13
time = 12:07:24
latitude = 63.976
longitude = -21.949
depth = 1.1
size = 0.6
quality = 58.73

just now scraping from web site Iceland Met 9:04:
rij = 't':new Date(2017,10-1,19,6,11,52),'a':'0.80247','lat':'66.530','lon':'-17.957','dep':'5.7','s':'1.4','q':'52.97','dL':'2.7','dD':'ESE ','dR':'Grmsey'
date in the later is 19-10-2017 6:11:52

copy past from website

Date	Time	Latitude	Longitude	Depth	Magnitude	Quality	Location
Thursday19.10.2017	06:11:52	66.530	-17.957	5.7 km	1.4	52.97	2.7 km ESE of Grímsey

[koddsson]

That sounds weird @Burgje and makes me think we are scraping two different sites. As far as I can tell we here scrape http://hraun.vedur.is/ja/skjalftar/skjlisti.html which has the latest earth movement on 2017-10-13. Are you scraping that same data source?

[Burgje]

I scrape from http://en.vedur.is/earthquakes-and-volcanism/earthquakes/#view=table
but it takes some manipulation to get the data in the right format. I take the HTML-code and extract the data which is in one line.

Using the API looked more promising as JSON is easier to handle.

[koddsson]

Ah! Now this all makes a bit more sense! Our data source is most likely not functioning right now! We should probably switch to the datasource you linked to.

[Burgje]

If I look at your scraping site http://hraun.vedur.is/ja/skjalftar/skjlisti.html it looks like the old icelandic Met site which might not be maintained anymore.

[koddsson]

Yup. We should switch to the one you posted.

[maxrahder]

I guess you know this, but the earthquakes feed hasn't shown new data for about a month now.

[Burgje]

I think it is realtively simple to do. I am not a java programmer, but if I look at the code it is getting the value's from the same line starting with "VI.quakeInfo =" and then doing some regex. Maybe somebody can give it a go?

[Larusrafn]

better source from the Skjalftavefsja:

http://drifandi.vedur.is/skjalftavefsja/identify.jsp?sEQLonRange=&sEQLatRange=&DateTimeRange=2017-12-4%2000:00:00;2017-12-6%2000:00:00&MagnitudeRange=0.0;9.0&DepthRange=0.0;100.0&Coords=&mapPosX=63.2321445206556&mapPosX2=66.6294620333688&mapPosY=-12.711430339621&mapPosY2=-25.3154783117871&orderby=OTIME&desc=true

just change the DateTimeRange to any 48 hour period (please do not use large time gaps, they might close this down on us if we exhaust the source)

skjalftavefsja: http://drifandi.vedur.is/skjalftavefsja/index2.html
to get the list, select Greina and then drag the square over the map area you are interested in. in the link above, the whole country is selected.

[MiniGod]

There's also this: http://hraun.vedur.is/ja/viku/sidasta/listi

It is linked to from vedur.is:
Jarðhræringar > Jarðskjálftayfirlit > Vikuyfirlit > Skjálftalisti

Thanks to @peturorri for pointing it out to me.

[Larusrafn]

This list is always a day old..... not a live update

[KristinnVikarJ]

I Fixed this in my PR #502

[GabeWater]

This is still an issue today. Most recent EQ is from 2017-10-13T12:07:24.000Z
When querying https://apis.is/earthquake/is