While Big Data enjoys widespread media coverage, not enough attention has been paid to what practitioners think — data scientists who manage and analyze massive volumes of data. We wanted to know, so Paradigm4 teamed up with Innovation Enterprise to ask over 100 data scientists for their help separating Big Data hype from reality. What we learned is that data scientists face multiple challenges achieving their company’s analytical aspirations. The upshot is that businesses are leaving data — and money — on the table.
This paper introduces a new benchmark, designed to test database management system (DBMS) performance on a mix of data management tasks (joins, filters, etc.) and complex analytics (regression, singular value decomposition, etc.)As a specific use case, we have chosen genomics data for our benchmark, and have constructed a collection of typical tasks in this area.
SciDB provides two closely related operations for creating intervals of ordered data for aggregation: windows and grids. This Tech Brief discusses useful functions in SciDB for computational finance.
For years, the quantitative finance industry has applied scientific computing techniques to computationally intensive problems like algorithmic trading and risk modeling. Continuing this trend, the industry can now capitalize on SciDB, whose capabilities support any number of quantitative finance applications at Big Data scale.This paper shows how.