Lifting the Fog into Space: Complementing Data Centers with Satellite On-Board Datacube Processing

In the ORBiDANSE project a datacube engine, rasdaman, has been ported to a cubesat, ESA OPS-SAT. Effectively, the satellite becomes a datacube service offering zero-coding query capabilities with the OGC Web Coverage Processing (WCPS) geo datacube analytics language and the ISO SQL/MDA (Multidimensional Arrays) SQL-embedded array analytics language.The datacube-enabled nanosat has been deployed, collecting data into an on-board datacube which has been queried successfully from ground. For the feasibility proof these queries included on-board cloud detection, histograms, timeseries analysis, edge detection, and more. In a preparatory step, the code was ported to Raspberry Pi because it contains a similar ARM processor as OPS-SAT. The port to the nanosat architecture was not straightforward and the lessons learnt have been cast into recommendations for payload architecture designers. Meantime, the codebases have been merged successfully, and the rasdaman codebase running on OPS-SAT is the same which normally is deployed in clouds (AWS, Google, etc.), supercomputing centers (such as the Taiwanese National High-Performance Computing Center and Forschungszentrum Jülich), and bare-metal servers (e.g. at Constructor University).This paves the way for on-board ad-hoc processing and filtering on Big EO Data, thereby unleashing them to a larger audience and in significantly shorter time. As such, the approach is substantially more powerful than deploying a trained neural network for some fixed task. In particular, the federation capabilities of rasdaman, in operational use for example in the Multipetabyte EarthServer federation, open revolutionary perspectives: not only can satellites form location-transparent federations which act as a single pool of information, but Earth and Space federations can be joined into a single planetary EO datacube resource. The capability of rasdaman to optimize analytics and fusion workloads distributed across highly asymmetric hardware allows combining cloud (such as data centers) and edge (such as satellites and drones) into a planetary fog computing setup.In our talk we will present the concept, its realization using rasdaman, the technical difficulties that had to be mastered, and the new avenues and opportunities that are being pursued next.ORBiDANSE was in part funded by the German Federal Ministry for Digital and Transport, OPS-SAT was funded by ESA.