Code and slides: Batched Key Access feature preview

At the moment, there are two big features in the works in MySQL optimizer - Subquery optimizations and Batched Key Access. While the former is a part of MySQL 6.0, I wrote about it here in my blog, and so forth, the latter was in nearly stealth mode until a couple of weeks ago.

That’s no longer the case:

  • Batched Key Access source code is now published as mysql-6.0-bka-preview tree.
  • Igor Babaev, the author of the feature, gave a talk about Batched Key Access at MySQL User Conference, and the slides are available here.
  • There is now Batched Key Access page on forge.mysql.com which a semi-official feature homepage which describes how you can get source, binaries, etc

If you’re developing any kind of “remote” storage engine or are using MySQL in a data warehousing application where disk IO is the bottleneck, I recommend to take a look.

Posted in joins on April 20th, 2008 by spetrunia | | 0 Comments

Slides: New subquery optimizations in MySQL 6.0 (new revision)

I’m now at MySQL Conference and have just finished my New Subquery Optimizations in MySQL 6.0 talk. If you’re eager to get the slides, here is the link to the pdf file and a mirror.

The talk has the same name as this MySQL University session but the slides have been thoroughly re-worked, and also there is new content:

  • More detailed explanation of subquery handling in 4.1/5.x
  • More reliable/justified benchmark numbers
  • Observations about subquery demographics
  • An attempt at comparative analysis of how MySQL’s subquery strategies compare to PostgreSQL’s
    • And a slide #28 that explains why we’re looking at PostgreSQL all the time

Have a nice viewing. And, if you get subqueries that should be fast but aren’t, we’re now actively looking at every subquery performance bug we get and will appreciate any input.

Posted in subqueries, how-it-works, benchmarks on April 16th, 2008 by spetrunia | | 0 Comments

On possibility to have more than 64 indexes

I was asked on Sun Tech Days whether it is possible to use more than 64 indexes in MySQL. I’ve asked my colleague, Ingo Strüwing, and found out the following: yes, it is possible to have more than 64 indexes, and no additional patches are required. You can get 5.0 source tarball, ./configure it with --with-max-indexes=128 and you’ll get a server that can use up to 128 indexes. One needs to be cautious though:

  • --with-max-indexes=N option looks like you could configure server with arbitrary maximum number of indexes. That’s not true, if you use N>128 then the server will compile but it will not even be able to start.
  • 128-indexes build of server is not widely used, so potentially it can be less stable than the regular build. My experience: I compiled current 5.0 source with --with-max-indexes=128 and it has passed all bundled tests (good) but I don’t see any tests that would use more than 64 indexes (bad). I’d recommend to do some testing before you roll out anything of value on 128-indexes build.
  • Work on 128-indexes version is documented here: BUG#10932.

Hope this information was helpful.

Posted in Uncategorized on April 5th, 2008 by spetrunia | | 4 Comments

Sun Tech Days St. Petersburg

I’m now at Sun Tech Days St Petersburg. MySQL got involved in it on a very short notice so there are no MySQL sessions or tutorials, and we only have a small booth. It’s decently manned though, there are six MySQLers here in total and we have no less than three people on the booth at nearly all times.

We organized some ad hoc sessions, so far, Alexey (aka Kaamos)’s talk about MySQL and dtrace attracted the most listeners, and Kostja was the second with his talk about new MySQL 5.1 features.

We get lots of questions on wide range of topics, the most popular are

  • Is Sun going to change MySQL license (close the source)? People are pleased to learn that MySQL will remain GPL.
  • How is MySQL development going to co-exist with PostgreSQL development inside Sun?
  • What, exactly, are MySQL licensing terms? People are not sure whether they will be able to use GPL or will need an OEM license, and what are the OEM licensing conditions and terms. It would be nice if MySQL website had more details about OEM agreements terms.

We get a fair number of technical questions as well. Here’s a quick dump in no particular order:

  • Does MySQL have special optimizations for Sun Niagara processors? What are the settings one should use when running on Niagara?
  • A complaint about poor subquery performance for a query generated by Hibernate. We figured that the issue is addressed in MySQL 6.0 by WL#3740 table pullout optimization.
  • Quite a few people are interested in in-memory transactional databases. Not all of them know about MySQL Cluster.
  • Most people are not aware of existence of MySQL Forge or MySQL University. People were interested to see slides, talks and other similar media, we’re handing out pieces of paper with MySQL Forge urls.
  • How does one create a load-balanced and/or fault-tolerant multi-machine MySQL installation? There is MySQL Proxy and there are 3rd-party solutions, there are also clustered JDBC drivers, but people would like to see a packaged or officially supported configurations. It would also be nice if there was an overview of available solutions and their relative properties.
  • MySQL’s command line client has poor tab completion in comparison to PostgreSQL’s. Indeed, hitting <TAB> in /usr/bin/mysql client always completes the list of tables and columns while in /usr/bin/psql completion is context-sensitive, e.g. if you type ALTER and hit TAB you’ll get a list of continuations available ALTER commands.
  • Is there a way to have more than 64 indexes?
  • Does MySQL’s GIS implementation allow to efficiently select primitives that are contained within (or have non-empty intersection with) a given axis-aligned rectangle? WL#3126 GIS: Precise spatial operations source tree needs to have some user-level documentation.
  • mysqldump and charsets. Quite a few people complained about having difficulties when doing export and import of databases in cp1251.
  • There needs to be some tool to assist upgrades. Several attendees said they were unable to upgrade their applications: the application stops working when moved to a newer MySQL version and they can’t figure out what went wrong. Ideally, they would like to have a tool that would just take their source code and update it with a newer MySQL, but that seems to be an “AI-complete” task. As a realistic option, we might use Proxy to run all queries against both old and new server, and report the differences.
  • MySQL cannot use index_merge over both fulltext and non-fulltext indexes, that is, it will use only one index (actually the fulltext one) for query like
    SELECT count(*) FROM tbl
    WHERE match(ft_key) AGAINST ('pizza' IN BOOLEAN MODE) AND non_ft_key='bar'

The conference is definitely worth to have presence on, also it’s nice to have enough people so we don’t have a strict booth watch schedule and there is always some rested force on the booth. Day-wise, the first day was the busiest, starting from the very opening hours. The second day had a slower start and even now (13:00) the crowd is not as thick. The good side of that is that it’s easier to have longer and uninterrupted discussions with interested attendees.

Posted in sun tech days on April 3rd, 2008 by spetrunia | | 2 Comments