Investigating the Influence of Asymmetric Error Bars on Atmospheric Retrievals

To produce spectra, MCMC (or similar) algorithms are used to constrain the transit depth for lightcurves in each wavelength channel. The central value of this fitting then serves as the data point in the spectrum and the spread of the distribution informs the error bars. An important feature to note here is that the distributions produced are not necessarily symmetrical; the upper and lower error bars may differ.In a retrieval, the different contributions to the atmosphere are constrained by iteratively forward modelling the transmission spectrum and assessing the agreement with the observed data using a likelihood function. It is common in the field to assume a Gaussian likelihood which takes two parameters: the centre of the Gaussian (the point in the spectrum) and its width (set by the error bars). Only one value can be supplied for the width and, as such, we are unable to account for any asymmetry in the error distribution under this framework.FIGURE 1: The observed transmission spectrum of WASP-39 b across the NIRSpec G395H instrument as reported by Carter et al., (2024). Panel a shows the data, marking any points with significant asymmetry with coloured crosses while panel b plots the asymmetry as a function of wavelength.With the improved quality of data supplied by the James Webb Space Telescope (JWST), in this work we seek to reaffirm our confidence in the Gaussian likelihood assumption by robustly testing a retrieval’s sensitivity to non-Gaussian effects in the observed data. We produce simulated transmission spectra using TauREx3, designed to mimic the observed data reported by Carter et al., (2024) for the planet, WASP-39 b (shown in Figure 1). In these datasets, both upper and lower error bars are reported and we observed a maximum asymmetry of ~77% between the upper and lower error bars across the G395H instrument. We artificially add non-Gaussian scatter to simulations and then run two types of retrievals on each dataset, one with a Gaussian and one with an asymmetric likelihood.Testing extreme cases of asymmetry, we find that the retrieval framework is sensitive to these non-Gaussian effects. Our two likelihoods find solutions in disagreement with each other which is cause for concern especially given that the differences are sometimes only visible in the joint posterior plots rather than the marginals for each parameter. However, when we scale back the errors and level of asymmetry to be representative of the current data, we find that the two retrievals achieve consistent predictions.FIGURE 2: Results of a Gaussian likelihood (purple) and an asymmetric likelihood (green) retrieval run on a simulation with +200% asymmetric scatter randomly added to every point in the spectrum. In this case, the asymmetric likelihood distribution matches the scattering distribution exactly. True values for the simulation are shown in black.The outcome of these investigations is that the Gaussian assumption is safe given the current quality of data and observed levels of asymmetry but caution is warranted when the level of asymmetry is greater. In the latter cases, not only is the level of asymmetry between the upper and lower error bars important, but also, the precise shape of the asymmetric distribution. In some tests, we observe an over-compensation by our asymmetric likelihood due to different distributions used to add noise to and to retrieve the parameters of the spectrum. This justifies the need for more precise reporting of lightcurve fittings beyond only providing the two or three summary statistics used to fit the spectrum.