In quantitative finance, a sophisticated model system could consist of a large collection of supervised ML models. Each ML model may require hundreds of features, many of which are transformed from “raw” input data columns using PySpark. Managing consistency of these transformations across hundreds of ML models in an integrated model system can be a daunting task, let alone the evolution of these transformations in model research and development. A domain specific language for expressing such transformations was invented to provide not only a much more clean and succinct grammar, but also a structured specification that can be automatically scanned for human errors such as cyclic definitions, conflicts, typos, etc. It greatly enhances the efficiency and productivity of model development.
CNN model for methane detection using remote sensing imagery
A convolutional neural network (CNN) model was built to predict trace-gas concentration from remote sensing images for the first time.
A remote sensing system with multi-spectral or hyper-spectral capabilities can be used to identify materials and their compositions on the ground from a few miles (airplanes) to hundreds of miles (satellites) away in the sky. It could be also used to detect trace gas (such as methane, an important greenhouse gas) near the ground if condition is right.
Such detection relies on the individual spectrum for each pixel in the multi- or hyper- spectral remote sensing image. Trace gas like methane has a signature absorption spectrum that sometimes can be identified from remote-sensing spectra. As a result, the methane concentration could be obtained for each single-pixel spectrum, and an image of methane concentration can be derived accordingly. This is the pixel-wise approach. However, the result of this pixel-wise approach can be quite noisy and has very large error bars due to various reasons such as low reflectivity of some ground covers, and fluctuation in detector sensitivity.
A CNN model can be used to mitigate some of these problems by combining information from neighboring pixels. However, to train such a model, a large amount of data needs to be available for the supervised learning, while remote sensing data with ground truth of trace gas concentration is scarce and it is very expensive to acquire large amount of training data.
To overcome the data problem, we combined micro-scale meteorology modeling and atmospheric radiative transfer simulation to obtain synthetic satellite images. This allows us to build sufficient amount of training, validation, and testing images for our CNN model which helps facilitate methane detection. This novel, CNN-based method improves greatly the robustness of methane detection, compared to the traditional pixel-wise approach.
This is a satelytics project.
ripp
ripp stands for Remote-sensing Imagery Processing Pipelines. It
- is a python framework for remote sensing imagery processing
- allows local and remote (AWS) workflows using CLI and a yaml configuration file
- facilitates orthoregistration and registration of satellite images
- facilitates CNN workflow for methane detection using satellite imagery
This is a closed-source software package for satelytics.
DGSRES
The resolution function in direct geometry neutron chopper spectrometers are fascinating. They exist up to 4 dimension in powder and single crystal experiments.
- Web app
- Experimental data for ARCS and SEQUOIA
- Published ARCS data
- Source code
Monte Carlo neutron ray-tracing simulations for neutron scattering experiments
- A software framework for Monte Carlo ray-tracing simulation of modern neutron instruments, including novel sample/sample environments, and sophisticated detector systems.
- It contributes to the design of all 8 selected instruments in the first phase of the STS project. Relevant publications can be found here.
- ~570k lines of C++/Python.
Links
BraggEdgeModeling
Neutron Bragg edge modeling
iMars3D: Neutron tomography reconstruction
A snapshot of a neutron CT reconstruction.
Neutron has greater penetration depths and has different cross section dependency on elements than X-ray. Neutron tomography can be used to study a variety of larger samples and is finding applications in engineering, material science, biology, and archeology. This package is in use by ORNL neutron imaging beamlines for neutron CT reconstructions.
First automatic CT reconstruction for the CG-1D beamline at HFIR
- New metadata added to images acquired at CG-1D beamline to identify grouping of CT scans
- Auto-discovery of the image files improved for a CT scan, including open beam and dark field images based on measurement time
- Auto-cropping algorithm improved to work with more measurement scenarios, especially those with slits
- New trigger from Cataloging service added to start the auto reduction service for CG1D
- Auto-reconstruction script implemented
- First successful auto-reconstruction happened Sept. 13, 2018
Multiphonon
Obtain phonon density of states from neutron DGS data, applying multiphonon corrections.
SEQUOIA detector calibration
Powder diffraction data were used to calibrate the position of detector packs for the SEQUOIA instrument.