INTERMAGNET - British Geological Survey

Description of web services provided by the Edinburgh GIN

Web applications

View INTERMAGNET observatory data: /GIN_V1/GINForms2

View INTERMAGNET data statistics: /GIN_V1/GINStatistics

View test observatory data at the Edinburgh GIN: /GIN_V1/GINTest2

View the Edinbugh GIN software version.

Web service

The web service allows access to INTERMAGNET data in the form of data files in community formats, interoperable formats (such as JSON) and plot files. Metadata and statistics on data provision and data usage are also available. A full description is given here so that other organisations may easily incorporate these services into their own products. The type of information that the services provides is controlled by parameters that are passed to the service in the URL's query string.

The web service is available from this base URL: /GIN_V1/GINServices. The services support both POST and GET http methods. The 'Request' parameter controls what information will be provided:

To help understand how to format URLs to retrieve data from the web service, a URL generation application is available. This application allows you to enter values for the data that you want to retrieve and then to see (and copy) the URL that can be used to retrieve the data.

Request Description
GetCapabilities Get lists of the things that this service is able to provide.
GetData Get geomagnetic data, either in the form of data files, or magnetograms.
GetDataDirectory Get a directory of the available geomagnetic data for a given set of observatories and time window.
GetDataLagTime Get a listing of the lag time (the amount of time between the current time and the time of the latest data sample from an observatory). The lag time is sampled hourly for each observatory and recorded to allow operators to see how long their data is taking to reach the GIN.
GetDownloadScript Get a script (or batch file) that can be used to download data. The script makes use of the 'GetData' service to retrieve geomagnetic data.
GetMonthlyDataStatistics Get the statistics files that are sent to observatories each month to describe the amount of data that has been received.
GetUsageStatistics Get the statistics on data downloads from the web and ftp sites.

Capabilities service

The capabilities service is designed to allow remote systems to interrogate the GIN and discover its capabilities. The service returns:

  • A list of the observatories known to the GIN.
  • A list of valid data types.

The following parameters may be used to control the capabilities request:

Parameter Description Valid values Default value
Request Must be set to GetCapabilities
Format The format that capabilities will be sent in. excel; html; json; xml xml
testObsys Whether to use the GIN's test observatories (set parameter to '1') or production observatories 1; 0 (but see test and production observatories in the embargo section). 0

Use the Web Service URL Generator to see how to create URLs for this service.

Data service

The data service provides geomagnetic data from INTERMAGNET observatories either as data files or as magnetograms. The following parameters may be used to control the data request:

Parameter Description Valid values Default value
Request Must be set to GetData
Format The format that the data will be sent in, including data file and magnetogram formats.
  • Data files: covJson; excel; html; xml; json; iaga2002; imagcdf; imfv122; wdc.
  • Magnetograms: png, jpg, pdf.
xml
testObsys Whether to use the GIN's test observatories (set parameter to '1') or production observatories 1; 0 (but see test and production observatories in the embargo section). 0
observatoryIagaCode Three digit IAGA observatory code specifying which observatory to retrieve data for. Any of the observatory codes returned using the GetCapabilities request. No default - this value MUST be specified.
samplesPerDay The sample rate of the required data, specified in samples per day. The special codes 'Minute' (for 1440 samples per day) and 'Second' (for 86400 samples/day) are also recognised. Currently limited to 1440 (Minute) and 86400 (Second). Minute.
dataStartDate The start date for data. A date in the form yyyy-mm-dd. Yesterday
dataEndDate The end date for data. A date in the form yyyy-mm-dd. One day after data start date.
dataDuration An alternative to dataEndData - the length of data. An integer number of days. 1
publicationState The data's 'publication state' or data type.
  • "adj-or-rep" - the best data available
  • "definitive" - definitive data
  • "quasi-def" - quasi-definitive data
  • "adjusted" - provisional (also called adjusted) data
  • "reported" - variometer (also called reported) data
  • "test" - test data
There is more information on data types in the INTERMAGNET technical reference manual: https://intermagnet.github.io/docs/Technical-Manual/technical_manual.pdf .
adj-or-rep (best available) when testObsys==0; test (only valid when testObsys is set to 1)
recordTermination The type of record termination used with some data files. Only applies to data in the format iaga2002, imfv122 or wdc Windows; UNIX UNIX
traceList Only applies to magnetogram plots. This parameter should describe the list of traces to display. Each code in the list will add a new trace to the display. Codes are a single character.
  • 1 (first recorded component)
  • 2 (second recorded component)
  • 3 (third recorded component)
  • 4 (fourth recorded component)
  • X (geographic X component)
  • Y (geographic Y component)
  • Z (vertical component)
  • H (horizontal component)
  • D (declination), I (inclination)
  • F (total field calculated from vector data)
  • S (F total field from a scalar instrument)
  • C (difference in scalar and vector total field)
1234
colourTraces Only applies to magnetogram plots. Allows colouring of the magnetogram. true (for colour); false (for greyscale) true
pictureSize Only applies to png and jpg magnetogram plots. Sets the size of the plot, in pixels. A string in the form 'XXXxYYY' or 'XXX,YYY' or 'XXX YYY', where X is the width in pixels and Y is the height. Alternatively set to 'Automatic' to let the software calculate a reasonable value. Automatic
pdfSize Only applies to pdf magnetogram plots. The size of the PDF plot, in centimetres. Set to a string in the form 'XXXxYYY' or 'XXX,YYY' or 'XXX YYY', where X is the width in cm and Y is the height. 21.0x29.7 (A4)
dataScale Only applies to magnetogram plots. The vertical scale value for the plots Set to a string in the form 'XX nT / YY minutes' where XX is the height of the trace in nT (for field value traces) and YY is the height of the trace in minutes of arc (for angular value traces). Alternatively set this to 'Automatic' to allow the software to fit the plot to the size of the trace. Automatic

Use the Web Service URL Generator to see how to create URLs for this service.

Data directory service

The data directory service gives precise information on the amount of geomagnetic data available to the user. The following parameters may be used to control the data request:

>
Parameter Description Valid values Default value
Request Must be set to GetDataDirectory
Format The format that the data will be sent in. Available formats are: plain; html; json; xml. The plain format is similar to the format used with the previous e-mail data-directory service. xml
testObsys Whether to use the GIN's test observatories (set parameter to '1') or production observatories 1; 0 (but see test and production observatories in the embargo section). 0
observatoryIagaCodeList A comma separated list of three digit IAGA observatory codes specifying which observatories to retrieve directory information for. Any of the observatory codes returned using the GetCapabilities request. No default - this value MUST be specified.
samplesPerDay The sample rate of the required data, specified in samples per day. The special codes 'Minute' (for 1440 samples per day) and 'Second' (for 86400 samples/day) are also recognised. Currently limited to 1440 (Minute) and 86400 (Second). Minute.
dataStartDate The start date for data. A date in the form yyyy-mm-dd. Yesterday
dataDuration The length of data. An integer number of days. 1
publicationState The data's 'publication state' or data type.
  • "adj-or-rep" - the best data available
  • "definitive" - definitive data
  • "quasi-def" - quasi-definitive data
  • "adjusted" - provisional (also called adjusted) data
  • "reported" - variometer (also called reported) data
  • "test" - test data
There is more information on data types in the INTERMAGNET technical reference manual: https://intermagnet.github.io/docs/Technical-Manual/technical_manual.pdf .
adj-or-rep (best available) when testObsys==0; test (only valid when testObsys is set to 1)
optionsA comma separated list of options. Any option may be prefixed with 'no' to turn the option off (so for example, 'noheader' is a valid option).
  • header - only applies to plain text format - controls printing of the header
  • showgaps - prints detailed information on gaps (showing their start and end times
  • separationline - only applies to plain text format - controls printing of a line between each observatory.
   

Use the Web Service URL Generator to see how to create URLs for this service.

Data lag time service

The data lag time service shows the difference between current time and the time of the most recently recorded sample from an observatory. The lag time is sampled once an hour for each observatory that sends data to the Edinburgh GIN. The lag time service allows operators to plot individual lag time figures for an observatory, also a monthly summary of lag times for all observatories. The following parameters may be used to control the data request:

>
Parameter Description Valid values Default value
Request Must be set to GetDataLagTime
Format The format that the data will be sent in. Available formats are: xml, json, png, jpg, pdf. xml
testObsys Whether to use the GIN's test observatories (set parameter to '1') or production observatories 1; 0 (but see test and production observatories in the embargo section). 0
observatoryIagaCodeList A comma separated list of three digit IAGA observatory codes specifying which observatories to retrieve directory information for. Any of the observatory codes returned using the GetCapabilities request. An empty list, indicating that all observatories should be included in the returned results.
samplesPerDay The sample rate of the required data, specified in samples per day. The special codes 'Minute' (for 1440 samples per day) and 'Second' (for 86400 samples/day) are also recognised. Currently limited to 1440 (Minute) and 86400 (Second). Minute.
dataStartDate The start date for data. A date in the form yyyy-mm-dd. Start of the previous month
dataDuration The length of data. An integer number of days. Number of days in the month
thresholdA maximum value for the lag time - lag times larger than this value will be truncated to this value. The purpose of this option is to prevent very large lag times (which result from a station not sending data) from 'swamping' the plots, which are autoscaled. An integer number of minutes 72 x 60, ie 3 days, which is the time within which INTERMAGNET mandates that an observatory should transmit its data to the GIN.
chartType Only applies to plots. A bar chart is useful when comparing lag times across a number of observatories, whereas a line chart is more useful to see detailed data from a few observatories. Set 'bar' or 'line'. Bar
pictureSize Only applies to png and jpg magnetogram plots. Sets the size of the plot, in pixels. A string in the form 'XXXxYYY' or 'XXX,YYY' or 'XXX YYY', where X is the width in pixels and Y is the height. Alternatively set to 'Automatic' to let the software calculate a reasonable value. Automatic
pdfSize Only applies to pdf magnetogram plots. The size of the PDF plot, in centimetres. Set to a string in the form 'XXXxYYY' or 'XXX,YYY' or 'XXX YYY', where X is the width in cm and Y is the height. 21.0x29.7 (A4)

Use the Web Service URL Generator to see how to create URLs for this service.

Download script service

Providing bulk data downloads is a challenge for web service providers. Allowing too much data to be downloaded in one go can impact the performance of a web server. This system solves the problem by creating download scripts, which are retrieved by the user then run (on either UNIX or Windows) to download the data required by the user. The script is designed in such a way as to ease the burden on the web server. Because of this, no restrictions are placed on the amount of data that the user can download. Parameters fully control the data that the user requires - the download script will be created to allow retrieval of the data as specified in the parameter set. Further details are available on the download page of the interactive form.

Parameter Description Valid values Default value
Request Must be set to GetDownloadScript
Format The operating system that the script should be designed for. This also controls the record termination type for the data files that will be downloaded. Windows; UNIX Windows
testObsys Whether to use the GIN's test observatories (set parameter to '1') or production observatories 1; 0 (but see test and production observatories in the embargo section). 0
observatoryIagaCodeList Three digit IAGA observatory code specifying which observatories to retrieve data for. This parameter may be used more than once to specify as many observatories as are required. Any of the observatory codes returned using the GetCapabilities request. No default - this value MUST be specified.
samplesPerDay The sample rate of the required data, specified in samples per day. The special codes 'Minute' (for 1440 samples per day) and 'Second' (for 86400 samples/day) are also recognised. Currently limited to 1440 (Minute) and 86400 (Second). Minute.
publicationState The data's 'publication state' or data type.
  • "adj-or-rep" - the best data available
  • "definitive" - definitive data
  • "quasi-def" - quasi-definitive data
  • "adjusted" - provisional (also called adjusted) data
  • "reported" - variometer (also called reported) data
  • "test" - test data
There is more information on data types in the INTERMAGNET technical reference manual: https://intermagnet.github.io/docs/Technical-Manual/technical_manual.pdf .
adj-or-rep (best available) when testObsys==0; test (only valid when testObsys is set to 1)
dataStartDate The start date for data. A date in the form yyyy-mm-dd. Yesterday
dataDuration The length of data. An integer number of days. 1
downloadDataFormat The format that the downloaded data will be sent in. iaga2002; imagcdf; imfv122; wdc.
iaga2002
folderOptions Options for the directory hierarchy to create to store the data in.
  • YearOnly - arrange all data from all observatories by year.
  • ObservatoryOnly - arrange all data from all years by observatory.
  • ObservatoryThenYear - put data into year directories under observatory directories.
  • YearThenObservatory - put data into observatory directories under year directories.
  • NoFolders - do not use any directories to organise the data.
YearThenObservatory
proxyAddress Address of the local proxy server. An address in the form [username:password@]address[:port] where username and password are the optional credentials needed to authenticate with the proxy server, address is the INTERNET name of the proxy server and port is the port address to use on the proxy server. Don't use a proxy server

Use the Web Service URL Generator to see how to create URLs for this service.

Monthly data statistics service

Statistics describing the amount of data received from each INTERMAGNET observatory are created on the 5th day of each month for the preceeding month. These statistics are emailed to all observatories and also archived. This service gives access to the archive. The archive consists of plain text files. Data are not available in any other format.

Each item in the archive has two dates:

  1. The date that the statistics refer to - a year and a month
  2. The date that the statistics were created on - a year, month and day. Typically this will be the 5th day of the month following the date that the statistics refer to, but it is possible that the statistics could be created on a different day, or that there could be multiple creation dates for the same statistics date.
Items in the archive are organised by their data type: reported; adjusted; quasi-definitive or definitive.

Parameter Description Valid values Default value
Request Must be set to GetMonthlyDataStatistics
Format The format that lists of statistics files will be sent in. The files themselves are always sent in text form. Lists of files can be sent in any of the formats listed json; txt txt
samplesPerDay The sample rate of the required data, specified in samples per day. The special codes 'Minute' (for 1440 samples per day) and 'Second' (for 86400 samples/day) are also recognised. Currently limited to 1440 (Minute) and 86400 (Second). Minute.
publicationState The 'publication state' or data type for which statistics are requested.
  • "definitive" - definitive data
  • "quasi-def" - quasi-definitive data
  • "adjusted" - provisional (also called adjusted) data
  • "reported" - variometer (also called reported) data
  • "test" - test data
There is more information on data types in the INTERMAGNET technical reference manual: https://intermagnet.github.io/docs/Technical-Manual/technical_manual.pdf .
reported
statisticsDate The date that the statistics refer to Dates in the form YYYY-MM If statisticsDate is missing, a list of all files for the given samplesPerDay and publicationState will be sent
creationDate The date that the statistics were created Dates in the form YYYY-MM-DD If creationDate is missing, a list of all statistics files for the given statisticsDate will be sent. Otherwise the file with the given statisticsDate and creationDate will be sent

Use the Web Service URL Generator to see how to create URLs for this service.

Data usage service

Daily and monthly statistics on data downloads from the web and ftp servers are available through the data usage service.

Parameter Description Valid values Default value
Request Must be set to GetUsageStatistics
Format The format that statistics will be sent in json
StatisticsDate The date that the statistics refer to. To retrieve a day file specify day, month and year. To retrieve a monthly summary specify a month and year only/ Dates in the form YYYY-MM or YYYY-MM-DD Last month

Use the Web Service URL Generator to see how to create URLs for this service.

Embargoing of data and plots

Data providers can specify an embargo period, during which data submitted to INTERMAGNET will not be distributed to users. The embargo period is specified in hours and applies to each observatory individually. Requests for data that is more recent than the embargo period will fail, showing the data as missing.

Organisations can apply to INTERMAGNET for a username/password to access this web service without the embargo being applied to their requests. Valid reasons for such a request include providing a service to the community that has been agreed with data providers, or an arrangement between a user and a data provider to supply data more rapidly than the embargo would allow.

To use a username provided for this purpose, send your http request using BASIC authentication ( https://en.wikipedia.org/wiki/Basic_access_authentication). If your authentication fails, the web service will not report the error and will continue as normal without authenticating the user (in which case the normal embargoes will apply).

The web sites at /GIN_V1/GINForms2 and /GIN_V1/GINTest2 do not currently support access via the username/password mechanism, and so will always apply any embargoes set by an institute on their observatory's data.

To set up an embargo for an observatory or to apply for a username and password contact us using the "comments" link at the end of this page.

Test and production observatories

Associated with each username is a flag which specifies whether to allow access to production observatories, test observatories or both. This flag is set up by the administrator at the time the username is created. The flag overrides the function of the 'testObsys' query string parameter.