Gnuastro 0.19 released
posted by Roy Schestowitz on Oct 24, 2022
Dear all,
I am happy to announce the availability of GNU Astronomy Utilities
(Gnuastro) version 0.19. It is packed with many exciting new features
and bug fixes (see below).
Gnuastro is an official GNU package, consisting of various
command-line programs, C/C++ library functions and Makefile extensions
for the manipulation and analysis of (astronomical) data. All the
programs share the same basic command-line user interface (modeled on
GNU Coreutils). For the full list of Gnuastro's library, programs, and
several comprehensive tutorials (recommended place to start using
Gnuastro), please see the links below respectively:
https://www.gnu.org/s/gnuastro/manual/html_node/Gnuastro-library.html
https://www.gnu.org/s/gnuastro/manual/html_node/Gnuastro-programs-list.html
https://www.gnu.org/software/gnuastro/manual/html_node/Tutorials.html
The full list of improvements in this version is available in the NEWS
file within the source, and also in [1] below. The new features are so
exciting/useful that Pedram (author of the newly added Warp features)
has recorded a video to show the tip of the iceberg in the Warp and
ConvertType programs. Please watch it to get a hands-on feeling of the
power of these new capabilities using SDSS and J-PLUS images (and go
the manual for the rest!):
https://peertube.stream/w/uq7SBDYZS1HRtJwCkbcDsz (17.5 minutes)
If you have any questions or ideas, or just want to stay up to date
with tips and other discussions, feel free to visit our Matrix chat:
#gnuastro:openastronomy.org
Here is the compressed source and the GPG detached signature for this
release. To uncompress Lzip tarballs, see [2]. To check the validity
of the tarballs using the GPG detached signature (*.sig) see [3]:
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.19.tar.lz (4.3MB)
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.19.tar.lz.sig (833B)
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.19.tar.gz (6.8MB)
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.19.tar.gz.sig (833B)
You can use a mirror for higher download bandwidth:
https://ftpmirror.gnu.org/gnuastro/gnuastro-0.19.tar.lz
https://ftpmirror.gnu.org/gnuastro/gnuastro-0.19.tar.lz.sig
https://ftpmirror.gnu.org/gnuastro/gnuastro-0.19.tar.gz
https://ftpmirror.gnu.org/gnuastro/gnuastro-0.19.tar.gz.sig
Here are the SHA1 and SHA256 checksums (other ways to check if the
tarball you download is what we distributed). Just note that the
SHA256 checksum is base64 encoded, instead of the hexadecimal encoding
that most checksum tools default to (see [4] on how to generate it).
For the list of software used to bootstrap this tarball, see [5].
fb53193886ca967a17bd8daf85835e8ad2b57780 gnuastro-0.19.tar.lz
f3UQsxNt2P9AxyVfe6DWLWn/3LU0OQoZc7w6+kkcWKQ gnuastro-0.19.tar.lz
adc6fcbf7ca476ffd3f6c4468527210ffeaff8b4 gnuastro-0.19.tar.gz
4bPNW0sSb/J34vSOit8BA9Z/wK0Hz5o9OqfgVSlDDjU gnuastro-0.19.tar.gz
For their direct contribution to this version's source code, I am very
grateful to Pedram Ashofteh-Ardakani (9 commits), Sepideh Eskandarlou
(6), Raul Infante-Sainz (6), Faezeh Bidjarchian (4), Jash Shah (3),
Marjan Akbari (2) and Elham Saremi (1). I am also grateful to (in
alphabetical order) to Marjan Akbari, Faezeh Bidjarchian, Sepideh
Eskandarlou, Giulia Golini, Raul Infante-Sainz, Teet Kuutma, Irene
Pintos Castro, Nafise Sedighi and Richard Stallman for their good
suggestions or reported bugs that have been implemented in this
release.
If any of Gnuastro's programs or libraries are useful in your work,
please cite _and_ acknowledge them. For citation and acknowledgment
guidelines, run the relevant programs with a `--cite' option (it can
be different for different programs, so run it for all the programs
you use). Citations _and_ acknowledgments are vital for the continued
work on Gnuastro, so please don't forget to support us by doing so.
Best wishes,
Mohammad
--
Staff Researcher
Centro de Estudios de Física del Cosmos de Aragón (CEFCA),
Plaza San Juan 1, Planta 2, Teruel 44001, Spain
[1] Noteworthy updates in Gnuastro 0.19 (released on October 24th, 2022)
** New features
Book:
- Two new sections added to the "General program usage tutorial" for
describing how to prepare a FITS image for high quality publication (in
PDF), and drawing vector graphics marks from a catalog over it (for
example to show your selected galaxies in the field of view). The use
the newly added features of ConvertType.
Arithmetic:
- Added new type operators and physical constants. All are also available
in Table's column arithmetic also).
- e: Base of the natural logarithm (no units).
- pi: Fraction of Circle cirumference to diameter (no units).
- c: speed of light in vaccume (in units of m/s).
- G: Gravitational constant (in units of m^3/kg/s^2).
- h: Plank's constant (in units of J/Hz).
- au: Astronomical Units (in units of meters).
- ly: Light years (in units of meters).
- avogadro: Avogadro's constant (in units of 1/mol).
- fine-structure: Fine structure constant (no units).
- counts-to-sb: convert counts to surface brightness (mag/arcsec^2).
- sb-to-counts: convert surface brightness (mag/arcsec^2) to counts.
- mag-to-sb: convert magnitudes to surface brightness over an area.
- sb-to-mag: convert surface brightness to magnitudes over an area.
- New operators that are specific to Arithmetic:
- collapse-median: collapse input dataset by calculating the median
along the given dimension.
- collapse-sigclip-std: Collapse with sigma-clipped standard deviation.
- collapse-sigclip-mean: Collapse with sigma-clipped mean.
- collapse-sigclip-median: Collapse with sigma-clipped median.
- collapse-sigclip-number: Collapse with number remaining after sigma-clip.
ConvertType:
- It is now possible to draw vector graphics marks from a catalog over
the output PDF images. The following options have been added to
ConvertType for doing this. See the "General program usage tutorial"
for a fully working example.
--marks: name of table containing mark information.
--markshdu: HDU of table if file given to '--marks' is FITS.
--markcoords: name or number of two columns containing coordinates.
--mode: if the coordinates are in 'img' (image) or 'wcs' (RA/Dec).
--markshape: name or number of column containing the shape of each mark. --markrotate: name or number of column containing rotation of each mark.
--marksize: name or number of column containing the size of the mark.
--sizeinpix: interpret the values in the size column as pixels.
--sizeinarcsec: interpret the values in the size column as arc-seconds.
--sizeinarcmin: interpret the values in the size column as arc-minutes.
--marklinewidth: name or number of column containing mark's line width.
--markcolor: name or number of column containing mark's color.
--listcolors: List all the 140 available colors, and show the colors on
24-bit (true color) terminal.
--marktext: name or number of column containing text under each mark.
--marktextprecision: number of decimals to print as text when the text
column (given to '--marktext') is floating point.
--markfont: name or number of column containing the font to use for the
the mark text (given to '--marktext').
--markfontsize: name or number of column containing the size of the
font to use for the mark text (given to '--marktext').
--showfonts: build a demo PDF with one page per font to show the
various available fonts on the system.
--listfonts: List the names of the available fonts on the terminal.
- It is now possible to select the color of the border of images produced
in vector graphics outputs (EPS and PDF) with '--bordercolor'.
Fits:
--pixelareaarcsec2: print the image pixel area in units of arcsec^2 to
standard output. Among other things, this is useful in creating a
surface brightness image using the new 'counts-to-sb' operator of
Arithmetic.
--pixelareaonwcs: Ouput an image with the same number of pixels as the
input. But each pixel's value shows its area on the sky (in
degrees-squared). This area is calculated after accounting for
distortion, projection or rotation. Implemented by Pedram
Ashofteh-Ardakani.
--edgesampling: extra sampling along each pixel's edge used to configure
the output of '--pixelareaonwcs', similar to Warp.
Statistics:
- Linear and Polynomial least squares fitting are now available and very
easy to call on the command-line. They are wrappers over the respective
least squares fitting functions of the GNU Scientific Library. The
interface is pretty simple, like the example below:
aststatistics table.fits -cX,Y,Yerr --fit=linear-weighted
It is also possible to estimate values and errors of the fitted model
on a new X axis. A complete example has been added to the newly added
"Least squares fitting" section of the book (under the Statistics
program documentation). Please see that tutorial to easily use this
feature. The following new options have been added to the Statistics
program for this purpose:
--fit: the model to use. Currently the following models are supported:
linear
linear-weighted
linear-no-constant
linear-no-constant-weighted
polynomial
polynomial-weighted
polynomial-robust
--fitweight: nature of the "weight" column (default: standard dev).
--fitmaxpower: maximum power of X in polynomial models.
--fitrobust: weight function to use in the "robust" polynomial model.
--fitestimate: File name, or number to estimate the fit on.
--fitestimatehdu: HDU containing table in file given to '--fitestimate'.
--fitestimatecol: Column containing X axis values for '--fitestimate'.
Table:
- It is now possible to customize the format of floating point numbers in
the plain-text outputs: when output is printed on the standard output
(command-line) or in plain-text files. The following new options have
been added for this new feature:
--txtf32format (or '-f'): Format of 32-bit floating point columns. This
can be either 'fixed' (for fixed-point notation) or 'exp' (for
exponential/scientific notation).
--txtf32precision (or '-d'): number of digits following the
decimal-point of 32-bit floating point columns.
--txtf64format (or '-p'): Format of 64-bit floating point columns. This
can be either 'fixed' (for fixed-point notation) or 'exp' (for
exponential/scientific notation).
--txtf32precision (or '-B'): number of digits following the
decimal-point of 32-bit floating point columns.
Warp:
- Can correct distortions (with any standard recognized by WCSLIB) and
simultaneously align the image to the coordinate system. When no named
linear operation (like '--rotate', '--scale' or etc) is requested, Warp
will go into this mode. It is highly customizable through the following
options. See the "Invoking Warp" section of the book for more. This
feature has been written by Pedram Ashofteh-Ardakani.
--center: RA, DEC of the center of the central pixel of output.
--width: Width of output in degrees or pixels (see '--widthinpix').
--widthinpix: interpret values of '--width' as pixels.
--cdelt: Pixel scale of output ('CDELTi' keywords in FITS).
--ctype: Coordinates and projection algorithm. Default: RA/Dec and
Gnomonic or 'TAN').
--edgesampling: extra sampling of pixel polygon to account for strong
non-linear projection or distortion effects, when necessary.
--gridfile: warp the input to the exact WCS and pixel grid of the file
given to this option. This is very useful when matching images from
differetn surveys. Using this option, you can also insert distortions
in an image (for example on a mock image, to make it match an
observed exposure with dithering+distortion).
--gridhdu: HDU containing image to be matched in '--gridfile'.
--checkmaxfrac: visualize the Moiré pattern of the warp in the
second extension of the output. This is the maximum fraction of
a single input pixel's area in the output pixel. When the output
pixel scale is similar to the input, the Moiré pattern can cause
varying artificial smoothing of the noise level. See the newly
added "Moiré pattern and its correction" section of the book for
more on its basics and how to reduce it in your outputs.
- List of WCS projections available in WCSLIB 7.12 (and therefore in
Gnuastro's Warp):
AZP: Zenithal/azimuthal perspective.
SZP: Slant zenithal perspective.
TAN: Gnomonic (tangential).
STG: Stereographic.
SIN: Orthographic/synthesis.
ARC: Zenithal/azimuthal equidistant.
ZPN: Zenithal/azimuthal polynomial.
ZEA: Zenithal/azimuthal equal area.
AIR: Airy.
CYP: Cylindrical perspective.
CEA: Cylindrical equal area.
CAR: Plate carree.
MER: Mercator.
SFL: Sanson-Flamsteed.
PAR: Parabolic.
MOL: Mollweide.
AIT: Hammer-Aitoff.
COP: Conic perspective.
COE: Conic equal area.
COD: Conic equidistant.
COO: Conic orthomorphic.
BON: Bonne.
PCO: Polyconic.
TSC: Tangential spherical cube.
CSC: COBE spherical cube.
QSC: Quadrilateralized spherical cube.
HPX: HEALPix.
XPH: HEALPix polar, aka "butterfly".
- List of WCS distortions available in WCSLIB 7.12 (and therefore in
Gnuastro's Warp):
TPD: Template Polynomial Distortion.
SIP: Simple Imaging Polynomial.
TPV: Polynomial distortion for Gnomonic (TAN) projection.
DSS: Digitized Sky Survey.
WAT: Chebyshev or Legendre polynomials for TNX and ZPX projections.
astscript-fits-view:
--ds9colorbarmulti: show a separate color-bar for each image in DS9. By
default this script will show a single color-bar for all the images to
help save space on the monitor when there are many images.
astscript-psf-stamp:
- sub-pixel warping is applied to ensure that your coordinate is at the
center of the central pixel of the output image. This results in a
_major_ improvement when estimating the center of the PSF.
--nocentering: disable sub-pixel warping when creating the PSF stamp. As
described above, the sub-pixel warping is critical for the central part
of the PSF, but for the outer parts it is statistically negligible. So
to avoid slowing down you pipeline, you can disable sub-pixel warping
with this option.
--snthresh: if given, the value to this option is assumed to be a
signal-to-noise ratio (S/N) threshold and all pixels below that S/N
will be masked. This is useful because we are often forced to stack
stars of differing magnitudes. While the fainter ones are good for the
inner parts of the star, they degrade the stack's outer parts. With
this option, the fainter stars won't harm the outer parts.
GNU Make extensions (in a Makefile)
It is now possible to use custom Gnuastro functions in GNU Make, using
its extension facilities with Dynamic libraries. GNU Make is a very
powerful workflow manager that is also used for data analysis (not just
for compilation). With the Gnaustro Make functions, (astronomical) data
analysis becomes even more easier and faster. In the following, you can
see the first set of such functions (they all begin with 'ast-'). For
more, see the newly added chapter in the Gnuastro manual.
- ast-version-is: will return '1' if the running Gnuastro has the given
version (argument of this function). This can be used to ensure
reproducibility in combination with Make's conditional features, see
the minimal working example in the manual.
- ast-text-contains: will return space-separated words within a larger
list that contain a certain string. The to-contain string can be
anywhere within the words of the larger list.
- ast-text-not-contains: will return space-separated words within a
larger list that DO NOT contain a certain string. The to-not-contain
string can be anywhere within the words of the larger list.
- ast-fits-with-keyvalue: takes a keyword name, a list of keyword
values, a HDU and a list of FITS files. It will return only those
FITS files that have the requested value(s) in the requested keyword
of the requested HDU.
- ast-fits-unique-keyvalues: takes a keyword name, a HDU and a list of
FITS files. It will return all the unique values given to that
keyword within the FITS files.
Library:
- GAL_CONFIG_HAVE_PYTHON: non-zero if Python3+Numpy features included.
- GAL_CONFIG_HAVE_GNUMAKE_H: non-zero if GNU Make extensions can be made.
- gal_box_border_rotate_around_center: width of box after rotation.
- gal_color_id_to_name: return the name of a color from its ID.
- gal_color_in_rgb: return the fraction of red-green-blue in a color.
- gal_color_name_to_id: return the ID of a color from its name.
- gal_dimension_collapse_median: collapse input along dim. using median.
- gal_dimension_collapse_sigclip_mean: collapse with sig-clipped mean.
- gal_dimension_collapse_sigclip_std: collapse with sig-clipped STD.
- gal_dimension_collapse_sigclip_median: collapse with sig-clipped median. - gal_dimension_collapse_sigclip_number: collapse with num. after sig-clip.
- gal_eps_shape_id_to_name: return the name of a shape from its ID.
- gal_eps_shape_name_to_id: return the ID of a shape from its name.
- gal_fit_name_to_id: Convert string name to ID of fitting model.
- gal_fit_name_from_id: Convert ID of fitting model to string name.
- gal_fit_name_robust_to_id: Convert name of robust weights to ID.
- gal_fit_name_robust_from_id: Convert ID of robust weights to name.
- gal_fit_1d_linear: linear fit of input columns.
- gal_fit_1d_linear_no_constant: linear fit with no constant.
- gal_fit_1d_linear_estimate: estimate a linear fit on a new X column.
- gal_fit_1d_polynomial: polynomial fit of input columns.
- gal_fit_1d_polynomial_robust: robust polynomial fit of input columns.
- gal_fit_1d_polynomial_estimate: estimate a polynomial fit on new X column.
- gal_fits_unique_keyvalues: extract all unique values to a certain
keyword in many files.
- gal_fits_with_keyvalue: select FITS image with a certain key value.
- gal_list_data_select_by_name: select a dataset from a list by its name.
- gal_list_str_cat: Concatenate (append) list to a space-separated string.
- gal_list_str_extract: Extract space-separated tokens to a list.
- gal_python_type_from_numpy: Convert Numpy's type id. to Gnuastro's.
- gal_python_type_to_numpy: Convert Gnuastro's type id. to Numpy's.
- gal_txt_contains_string: Check a certain string within in a larger one.
- gal_units_counts_to_sb: SB from counts, zeropoint and area.
- gal_units_mag_to_sb: surface brightness (SB) from magnitude and area.
- gal_units_sb_to_counts: counts from SB, zeropoint and area.
- gal_units_sb_to_mag: magnitude from SB and area.
- gal_warp_pixelarea: return image of same size, but with area on sky.
- gal_warp_wcsalign_init: initialize the WCS aligning structure.
- gal_warp_wcsalign_onpix: Per-pixel filling of output.
- gal_warp_wcsalign_onthread: function to give to pthreads.
- gal_warp_wcsalign: high-level function to align input by its WCS.
- gal_warp_wcsalign_free: free the contents of the WCS aligning structure.
- gal_wcs_free: free a WCS structure that is created or read by Gnuastro.
** Removed features
Statistics:
--refcol has been removed because it breaks the modularity principle
(given that it is the job of Gnuastro's Table program to limit rows
from a larger table based on many different criteria). The output of
Table can be directly piped to Statistics to achieve the same (and much
more feature-rich effect).
Warp:
--align: has been removed. This is because aligning an image (while
correcting for any possible distortion) is now the default behavior of
Warp (when no linear operations have been requested).
** Changed features
Book:
- The "General program usage tutorial" section is now the first section
of the Tutorial chapter, since it introduces the tools at a more basic
level. The "Sufi simulates a detection" (which was previously first)
has been moved to the fourth section.
Warp:
- The short format of the '--centeroncorner' option has been removed. The
'-c' is now the short format for the new '--center' option to Warp.
astscript-psf-scale-factor:
--widthinpix: new name for the old '--stampwidth' option. This was done
to have the same name to a similar option in Crop and help in
remembering.
astscript-psf-stamp:
--widthinpix: new name for the old '--stampwidth' option. This was done
to have the same name to a similar option in Crop and help in
remembering.
Library
- gal_eps_write: two new arguments have been added to draw marks, and to
set the border color.
- gal_pdf_write: similar to 'gal_eps_write'.
- gal_fits_hdu_open: new argument to optionally exit program if HDU
couldn't be opened.
- GAL_TABLE_DISPLAY_FMT_FIXED: new name for GAL_TABLE_DISPLAY_FMT_FLOAT
since it corresponds to the fixed-point notation of printing floating
points in plain-text (the '_FLOAT' suffix was too generic and unclear).
** Bugs fixed
bug #62861: '--printkeynames' of Fits program gets caught in an infinite
loop on FITS files that have empty keywords before
'END'. Found by Pedram Ashofteh-Ardakani.
bug #62892: Installed scripts don't account for differing LANG and
LC_NUMERIC. Found by Teet Kuutma and fixed by Raul
Infante-Sainz.
bug #62937: psf-scale-factor not being quiet, when requested. Found and
fixed by Sepideh Eskandarlou.
bug #62943: Couldn't read the value of width with '--snthresh' is called.
Found and fixed by Sepideh Eskandarlou.
bug #62944: No warning when the option value isn't immediately after the
equal sign in long format. Found by Faezeh Bijarchian.
bug #63013: Sigma clip segmentation fault when input has an integer type
with values close to saturation-level.
bug #63022: psf-scale-factor not saving the result in the output file,
found and fixed by Raul Infante-Sainz.
bug #63189: MakeProfiles custom profiles become NaN with a single row
being NaN, reported by Nafise Sedighi.
bug #63207: Match crashes when one input has no rows. Found by Sepideh
Eskandarlou.
bug #63257: Fits program's '--skycoverage' gives unreasonable outputs
when image crosses the RA=0 hour circle. Found by Irene
Pintos Castro.
bug #63261: Radial profile script ignores central pixel in azimuthal
profiles. Found and fixed by Sepideh Eskandarlou.
[2] Lzip has better compression ratio and archival features compared
to the `.gz' or `.xz' formats. Therefore Gnuastro's alpha/test
releases are only in this format, but for historical reasons we also
include `.gz' tarballs in the official releases. If you don't have
Lzip (you can check with `lzip --version' command), download and
install it from its webpage:
https://www.nongnu.org/lzip/lzip.html
If Lzip is present and you use GNU Tar, then the single command below
should uncompress and un-pack the tarball:
$ tar xf gnuastro-0.19.tar.lz
If the command above doesn't work, you have to un-compress and un-pack
it with two separate commands (or use a pipe to feed the output of the
first into the second: `lzip -cd gnuastro-0.19.tar.lz | tar -xf -'):
$ lzip -d gnuastro-0.19.tar.lz
$ tar xf gnuastro-0.19.tar
[3] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. First, be sure to download both the .sig file
and the corresponding tarball. Then, run a command like this:
gpg --verify gnuastro-0.19.tar.gz.sig
The signature should match the fingerprint of the following key:
pub rsa4096 2018-12-08 [SC]
52B0 4484 D806 C90D CB52 7249 71E8 9901 2D17 4B66
uid [ unknown] Mohammad Akhlaghi
If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.
gpg --recv-keys 71E899012D174B66
As a last resort to find the key, you can try the official GNU
keyring:
wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
gpg --keyring gnu-keyring.gpg --verify gnuastro-0.19.tar.gz.sig
[4] To get the base64 SHA256 checksum, you can use the command below
(assuming you already have the 'openssl' command-line program). Just
note that an extra '=' is printed as the last character that you
should ignore.
cat gnuastro-0.18.71-c982.tar.lz \
| openssl dgst -binary -sha256 | openssl base64 -A
[5] This tarball was bootstrapped (created) with the tools below. Note
that you don't need these to build Gnuastro from the tarball, these
are the tools that were used to make the tarball itself. They are only
mentioned here to be able to reproduce/recreate this tarball later.
Texinfo 6.8
Autoconf 2.71
Automake 1.16.4
Help2man 1.49.2
ImageMagick 7.1.0-51
Gnulib v0.1-5536-g0814a293a4
Autoconf archives v2022.09.03-4-gda89908
Read on