It is well known that implementing a data-driven application requires time and money. The result is that organizations, corporations and the government often lack the data analytics and management applications that they need since the budget limitations and the speed of business requirements do not allow for the needed applications to be built on time.

A major source of the cost of building of data-driven applications is the fact that the application has to collect data from multiple sources, combine them and provide them with the appropriate format to the visualization components. It is both a blessing and a curse of the Big Data era that this source of cost is increasing. The Big Data era is characterized by a big diversity of databases. Besides the customary SQL databases, nowadays data are also found on NoSQL, NewSQL and SQL-on-Hadoop databases. Furthermore, interesting data are also found on the client device (typically smartphone) and its browser. Finally, the plethora of visualization components creates the need for easily adjusting the results to the formats that the visualizations need.

Given the importance of semistructured data (such as JSON) both as input in NoSQL databases and as JSON (in particular) as the logical representation of the visualization input in MVVM architectures, the project created the SQL++ query language which accesses seamlessly both SQL and semistructured data. While the SQL++ idea (as an SQL extension for semistructured data) pre-existed, the project was focused on providing the full semistructured data functionality required - namely, account for the potential lack of schema, enable arbitrary inputs, outputs and transformations as powerful as the ones that XQuery achieved. Unlike XQuery this project was committed to produce an SQL compatible language, since this is what the majority of developers understands.

Per the original project objective, the project created a SQL++ distributed query processing engine, including the ability to refer to data on either the browser or server. Furthermore, the project addressed the case of live data, by developing Incremental View Maintenance that ensures that the views offered to the users are up-to-date, i.e., they reflect the state of the underlying databases.

The project expanded SQL++ into a Configurable SQL++ that formally captures the query language differences across the multiple semistructured query languages of the multiple NoSQL, newSQL and SQL-on-Hadoop databases of today. Thus the Configurable SQL++ became a useful tool for formally surveying the semantic differences between the multiple languages for NoSQL, NewSQL and SQL-on-Hadoop. We disseminated the Configurable SQL++ to the industry, in the order of creating a dialog and a common understanding that will eventually will lead to the appropriate (for NoSQL, NewSQL and SQL-on-Hadoop) clean and formal extensions for extensions of the SQL standard. The response of the database industry to SQL++ and Configurable SQL++ has been excellent. A number of NoSQL databases adopt and we anticipate that SQL++ will further influence the database industry.

Finally, Configurable SQL++ plays an internal role to the distributed query processor that accesses multiple sources of semistructured data. Recall, these sources are very diverse on the query languages that they use. Thus the distributed query processor needs to be able to interact with these diverse languages and rewrite the application requests into the languages they understand. Configurable SQL++ brings in a formal definition of the diversity aspects of the various languages. Then it becomes much easier for the distributed query processor to translate across these languages.

Last Modified: 08/11/2017
Modified by: Yannis Papakonstantinou