Paradigm4 White Papers

The Architecture and Motivation for SciDB
This paper presents a high level description of SciDB and explores the design of some of its critical components in detail.

Convolution is a database problem
A brief introduction to some of the algorithms involved in processing and analysis of very large image collections. Plus, a review two inter-related problem areas: image processing and deep learning.

Case Study: Managing Trillions with Custom Indexes
Paradigm4’s customer is a leading provider of research-based indexes and analytics. They chose SciDB to help them create custom descriptors from 40+ years of market observations on 75,000 securities. Learn how this industry leader uses SciDB to merge data with different time frequencies and improve scalabity and performance.

Case Study: Tick Database
A leading Chicago-based proprietary trading firm had bruising limitation in their long standing tick data store. By deploying SciDB the firm was able to improve performance 20X, improve trading algorithms, and allow quants and traders to self serve data.

Accelerating Bioinformatics Research with New Software for Big Data to Knowledge (BD2K)
This whitepaper describes where SciDB fits in the genomics and translational informatics data analysis pipeline and demonstrates use cases for data mining on genomics, phenotype, and outcomes data. We show how SciDB boosts bioinformatics analytical capabilities in support of research on disease understanding, biomarker discovery, and precision medicine.

Computational Finance with SciDB
For years, the quantitative finance industry has applied scientific computing techniques to computationally intensive problems like algorithmic trading and risk modeling. This whitepaper shows a representative—but by no means exhaustive—set of such problems, all solved at Big Data scale with SciDB.

SciDB for Bioinformatics AMI: Real-World Use Cases, Sample Queries, Real Data
SciDB is well-matched for managing, accessing, integrating, and analyzing diverse scientific data from ‘omics to outcomes. Our bioinformatics AMI makes it easy to test out SciDB in the cloud via Amazon EC2. This AMI includes data along with example bioassay and variant analysis use cases to let you quickly try out SciDB. (Redirects to Try SciDB Page for Bioinformatics AMI)

Windowed Aggregates in SciDB
SciDB’s array model is ideally suited for storing, managing, and analyzing financial time series. Windowed aggregates performed natively in SciDB provide an efficient and versatile way of converting raw data into a format suitable for exploratory data analysis, algorithmic trading system development and downstream modeling workflows.

Leaving Data on the Table: Data Scientists Reveal Obstacles to Big Data Analytics
While Big Data enjoys widespread media coverage, not enough attention has been paid to what practitioners think — data scientists who manage and analyze massive volumes of data. We wanted to know, so Paradigm4 teamed up with Innovation Enterprise to ask over 100 data scientists for their help separating Big Data hype from reality. What we learned is that data scientists face multiple challenges achieving their company’s analytical aspirations. The upshot is that businesses are leaving data — and money — on the table.