Header Image - Computational Scientist

Tag Archives

8 Articles

Domain Specific Language for “Variable Transformation” with PySpark

by Jiao 0 Comments

In quantitative finance, a sophisticated model system could consist of a large collection of supervised ML models. Each ML model may require hundreds of features, many of which are transformed from “raw” input data columns using PySpark. Managing consistency of these transformations across hundreds of ML models in an integrated model system can be a daunting task, let alone the evolution of these transformations in model research and development. A domain specific language for expressing such transformations was invented to provide not only a much more clean and succinct grammar, but also a structured specification that can be automatically scanned for human errors such as cyclic definitions, conflicts, typos, etc. It greatly enhances the efficiency and productivity of model development.

CNN model for methane detection using remote sensing imagery

by Jiao 0 Comments
CNN model for methane detection using remote sensing imagery

A convolutional neural network (CNN) model was built to predict trace-gas concentration from remote sensing images for the first time.

A remote sensing system with multi-spectral or hyper-spectral capabilities can be used to identify materials and their compositions on the ground from a few miles (airplanes) to hundreds of miles (satellites) away in the sky. It could be also used to detect trace gas (such as methane, an important greenhouse gas) near the ground if condition is right.

Such detection relies on the individual spectrum for each pixel in the multi- or hyper- spectral remote sensing image. Trace gas like methane has a signature absorption spectrum that sometimes can be identified from remote-sensing spectra. As a result, the methane concentration could be obtained for each single-pixel spectrum, and an image of methane concentration can be derived accordingly. This is the pixel-wise approach. However, the result of this pixel-wise approach can be quite noisy and has very large error bars due to various reasons such as low reflectivity of some ground covers, and fluctuation in detector sensitivity.

A CNN model can be used to mitigate some of these problems by combining information from neighboring pixels. However, to train such a model, a large amount of data needs to be available for the supervised learning, while remote sensing data with ground truth of trace gas concentration is scarce and it is very expensive to acquire large amount of training data.

To overcome the data problem, we combined micro-scale meteorology modeling and atmospheric radiative transfer simulation to obtain synthetic satellite images. This allows us to build sufficient amount of training, validation, and testing images for our CNN model which helps facilitate methane detection. This novel, CNN-based method improves greatly the robustness of methane detection, compared to the traditional pixel-wise approach.

This is a satelytics project.

Monte Carlo neutron ray-tracing simulations for neutron scattering experiments

by Jiao
Monte Carlo neutron ray-tracing simulations for neutron scattering experiments

MCViNE: 

  • A software framework for Monte Carlo ray-tracing simulation of modern neutron instruments, including novel sample/sample environments, and sophisticated detector systems.
  • It contributes to the design of all 8 selected instruments in the first phase of the STS project. Relevant publications can be found here.
  • ~570k lines of C++/Python.

Links

iMars3D: Neutron tomography reconstruction

by Jiao
iMars3D: Neutron tomography reconstruction

A snapshot of a neutron CT reconstruction.

Neutron has greater penetration depths and has different cross section dependency on elements than X-ray. Neutron tomography can be used to study a variety of larger samples and is finding applications in engineering, material science, biology, and archeology. This package is in use by ORNL neutron imaging beamlines for neutron CT reconstructions.