I discussed the issue of RBSI and CBSI implementation with some engineers. These are further practical views.

First, we know that Row Based Storage is very effective for OLTP applications. Want to kill Column Based Storage sometime, just do a select * on your typical 60+ column trading table (using a where clause to say fetch 1,000 rows). We all know ways of killing Row Based Storage. I am not trying to knock either, but each are firmly entrenched at opposite ends of the spectrum. Again it does not meant that the thought of putting them together is not attractive – everyone can see some of the advantages here…..having flying cars or even flying Ferraris would be a great way to relieve congestion too – and the technology exists today, but it has to be tempered with a bit of reality in what is practical today, given the economics and infrastructure….which brings up the second point. Arguably, however, the idea of merging the two in a single instance suggests that “One Size Could Fit All” – which is an argument that ignores operational environmental considerations

Secondly, the complexity of (effectively) putting the Sybase IQ style access methods (CBSI) let alone optimization (i.e. costing a CBSI query is different from costing a RBSI query), index maintenance (IQ penalizes loads to maintain stats and focuses more on bulk loads as a result – picture the overhead of an OLTP system that is forced to maintain stats on certain tables at insert time)….not saying all this *could not* be done – but it would be a complete top to bottom re-write of say RBSI such as ASE or Oracle that would effectively kill? any other project (RAC, SDC, etc.) as well as most other RBSI enhancements for multiple years. That is the practical/infrastructure problem that the engineers point out.

The current way forward is to have both engines working as separate entities. If you think about it, this is not too different from what MySQL tried to do – they separated storage logic from query logic and allowed you to pick the storage engine. The *current* direction for Sybase as I know is a twist on that – allow multiple query engines to access multiple storage engines – whether that is a federated ASE -> ASE or a OLTP/DSS ASE -> Sybase IQ (and in reverse)

Thank you for your clarification. I will pick up on your terminologies.

As you correctly pointed the physical implementations of the relational model are an entirely different matter, and do indeed vary according to the various vendor’s offerings. Some implement the relational model (objectively depending on your familiarity/attachment) better than others. I call Sybase ASE or Oracle a row based storage implementation (RBSI) of Relational Model (RM) and Sybase IQ a column based storage implementation (CBSI) of RM.

Let me for focus on the back-end stuff, I believe a vendor like Sybase and I am sure Oracle as well has both technologies for both RBSI and CBSI in hand. To make RBSI and CBSI implementations work seamlessly is a challenge. Every vendor recognises the need to create an RBSI and CBSI solution.

Clearly there is a need for a physical implementation that focuses on better retrieval of data for Read Intensive activities. To believe that one RBSI model of vendor has cracked it much better than the other ones is simply naive and at best a marketing ploy.

I would envisage a scenario in the future where a vendor like Sybase will retain ASE but will incorporate column based storage implementation into it. That is probably the best way of taking this forward. Imagine one has the ability to create a table with row or columned based storage option:

CREATE TABLE … STORAGE

With that in mind the challenge for the designers would be how to make the RBSI Execution Engine work in tandem with the CBSI Execution engine when joining a row based storage table with a column based storage table! I can see tremendous potential with partitioned tables as well. You could create the most active partition with row storage and the other partitions with column storage and thus compressed, resulting in considerable saving of space. The potential is infinite and so is the challenge. I am sure the technology is achievable.

Thank you for your notes. Can I just clarify that I do work on Oracle equally as well and I am fond of it. It is natural to expect emotions to run high when religious arguments like databases come into it. This happens in both Oracle and Sybase forums so I have got used to it. However, the issue at stake is whether the traditional transactional base relational models like Oracle and Sybase can do as good a job compared to non traditional models like column based storage databases such as Sybase IQ, when it comes to read intensive activities. I have to emphasis here that both the designs that I mentioned are based on relational model (i.e. Codd’s original ideas), just the way the data stored in different. It is becoming increasingly obvious that the traditional models have limitations in this arena. That is all. In summary you can only take the traditional RDBMSs so far in read intensive area.

“Sybase Transact SQL Programming Guidelines and Best Practices: A Practitioners Approach Through Example”

He also has a book coming out soon that compares and contrasts Oracle and Sybase.

Anyway, I just wanted to clear the air about that point.

Cheers, Tim

I like the “probably”! As if Mich has an entire Alladdin’s cave of wonderful ideas to smash the RDBMS approach used by all the big players.

Mich: 10 out of 10 for sheer gall, but 0 out of 10 for real-world value.

2. I have used Oracle for data warehousing for more that 13 years, it does the job .. so what the issue, apart from cost?

3. I have worked in investment banking for more than twenty years, the last thing that banks want is traders seriously adopting copycat trading behaviours. If you know anything about trading you will know why.

4. Independent Sybase consultant? hahahaha … Sybase is a good product .. but …