Adding data to the Demeter standards database

You can examine all the files associated with Demeter's standards database in the git repository: https://github.com/bruceravel/demeter/tree/master/lib/Demeter/share/standards

This is not an especially robust system. It was built assuming that any entries in the database have been vetted by a knowledgeable expert and that adequate information will be provided for each entry in the database.

Keywords in the standards.ini file

Here are all of the keywords that may be used in a database entry in the standards.ini file:

element

The two letter symbol for the absorbing element. This can be omitted if (and only if) the identifier of the database entry can be interpreted as an element symbol (see the Cr foil example below)

tag

a brief textual description of the data -- usually the chemical formula of the name of the compound

comment

a lengthier (but still one-line) description of how, where, and by whom the data were measured

crystal

monochromator crystal type

file

the location on disk or on the web of the file. The token %share% indicates the lib/Demeter/share directory under the place where Demeter is installed and is used to make the database location independent (as long as it is part of a correctly installed Demeter installation)

record

if the file is an Athena project file, this indicates the record number of these data. If the data is from a column data file, there must not be a record entry.

xmu

if this is set to 1, the specified column data file has already been converted from the beamline format into mu(E) data

energy

the column containing the energy array, expressed "gnuplot" style (e.g. $1)

numerator

the column containing the numerator array (I₀ for transmission data, I_f for fluorescence data), expressed "gnuplot" style (e.g. $2)

denominator

the column containing the denominator array (I_t for transmission data, I₀ for fluorescence data), expressed "gnuplot" style (e.g. $3)

ln

this should be 1 for transmission data and 0 for fluorescence or electron yield data

calibrate

this takes two numbers, the point in the energy array to use as e0 and the value to which that point should be calibrated

rebin

set this to 1 is these are column data that need to be rebinned

xanes

a list of energy values for annotation in mu(E). You should always include the zero-valent edge energy and the edge energy of these data as well as several interesting points

deriv

a list of energy values for annotation in deriv(mu(E))

Data formats

Column data files should be given a name suggestive of the contents and a file extension of .stan. Using that file extension helps Demeter's build system identify and correctly install those files. By convention, Athena project files contain data of a common absorber. That is not a necessary convention, but it helps keep things orderly.

Data in an Athena project file

This is preferred format for data in this database.

Data in an Athena project file are assumed to have been processed properly. Most importantly, the data have already been calibrated onto a proper absolute energy scale, presumably using a reference spectrum. An entry is recognized as coming from an Athena project file when the record keyword is set in the database entry. The calibrate keyword will be ignored for data from an Athena project file -- even if present. Thus it is essential that the data be properly calibrated in the project file.

The data should also already be rebinned (if it is the sort of data that requires that). Data recognized as coming from an Athena project file will not be rebinned when displayed by one of Demeter's standards utilities.

The energy, numerator, denominator, and ln keywords will be ignored as well for Athena project data.

Here is an example:

[mn oxyhydroxide]
element     = mn
tag         = MnOOH
comment     = gamma-MnOOH (3+) powder measured at APS 10ID in June 2005 Shelly Kelly, Max Boyanov, Ken Kemner, and BR
crystal     = Si(311)
file        = %share%/standards/data/Mn.prj
record      = 9
xanes       = 6539, 6549.58, 6561.1, 6574.46
deriv       = 6539, 6549.58, 6556.24, 6572.03

This is a normal INI type file. It starts with an identifier in square brackets and is followed by key/value pairs which are separated by an equals sign and end at the end of the line. Blank lines can be interspersed with the keywords. The next entry starts with the next bracketed identifier.

It is important to choose the identifier and tag well. Ideally, these will be the chemical formula and compound name. Examine the standards.ini file for some examples of how I chose these values.

Credit should go where credit is due, so please identify the beamline and synchrotron where the measurement was made, the date of the measurement, and the persons involved in the measurement. Proper interpretation of the data, particularly issues related to resolution, requires knowing the crystal type. So always include this information.

The last two lines are the thing that makes this standards utility more useful than most. These data are annotated. Interesting points -- the edge energy, peaks and troughs of mu(E) or the derivative, interesting shoulders, etc -- get marked when the data are displayed. Chose four or five points each for mu(E) and the derivative. Choose particularly distinctive features and choose only features within about 100 volts of the edge. Look at how Demeter displays the annotated data to make sure that these labels don't overlap and are displayed prettily.

Data in a column data file

Here is an example of data contained in a column data file:

[cr]
tag         = Chromium foil
comment     = Chromium foil measured at NSLS X23A2 28/10/2008 by BR
crystal     = Si(311)
file        = %share%/standards/data/cr.stan
energy      = $1
numerator   = $2
denominator = $3
ln          = 1
rebin       = 0
calibrate   = 5989.4, 5989
xanes       = 5989, 5991.58, 5994.74, 6009.11, 6018.88
deriv       = 5989, 5991.58, 5995.03, 5998.31

Note that the record keyword is absent (as this is a column data file and not an Athena project file). Note also that this was measured as a normal step scan, so rebinning is turned off.

Demeter needs to know how to interpret these data, so the energy, numerator, denominator, and ln keywords are all specified, as is the calibrate keyword. In this case, the peak of the first derivative was found at 5989.4 eV. The calibrate line tells Demeter to shift the data by -0.4 eV and to set 5989 in the shifted data as the edge energy.

Data from the internet

Here is an example of fetching data from somewhere on teh internets:

[ni]
tag         = Nickel foil
comment     = Nickel foil, focused, detuned 50%, measured at APS 13ID June 2001
crystal     = Si(111)
file        = http://millenia.cars.aps.anl.gov/~newville/ModelData/ni_rt01.xmu
xmu         = 1
calibrate   = 8331.5, 8333
xanes       = 8333, 8336.6, 8358.2, 8382.7
deriv       = 8333, 8338.8, 8345.9, 8355.6

As before, information about how to calibrate and annotate the data are provided. Note that the xmu keyword is set, indicating that this column data file has already been converted from the beamline format to mu(E) data.

Incomplete data

Demeter can still use data even if it is not fully annotated. Here is an example:

[realgar]
element     = as
tag         = AsS Realgar
comment     = AsS powder on tape,  unfocused, detuned 75%, clockwise at E=12000eV, SSRL 2-3, 10K, April 1997
crystal     = Si(111)
file        = http://millenia.cars.aps.anl.gov/~newville/ModelData/as003a_1.xmu
xmu         = 1

In this case, the only annotation will be at the tabulated value of the absorber edge energy. No effort is made to calibrate these data as the calibrate keyword is omitted. Demeter will behave correctly if the tag and comment keywords are also omitted. However, if an entry is missing that much information, it probably is not suitable for inclusion in the database.

Extending the database

I gladly accept contributions. If you send me an Athena project file along with a text file containing database entries, I will gladly include them in the Demeter distribution.

You can add database entries system-wide by editing the lib/Demeter/share/standards/standards.ini file, which is found beneath the Demeter installation location. At the command line, do:

 perl -e 'use Demeter; print $INC{"Demeter.pm"}, $/;'

to figure out where that is.

I have not yet implemented it, but I also intend to import database entries from $HOME/.horae/standards.ini.