Andy Grove's Blog

Snow Leopard + IntelliJ issues

2009-09-15T15:28:00.002-06:00

Since upgrading to Snow Leopard and restoring Apple Java 1.5.1 from backup I've been having tons of stability issues with IntelliJ 8.1.3 and I finally got this resolved today. It seems that there is a reason why Apple chose to remove their 1.5.1 version from Snow Leopard after all. I think there are some compatibility issues with the new OS version.

Since modifying IntelliJ's startup configuration to use Java 1.6 instead of 1.5 and clearing out the old caches, all the stability issues seem to have gone away.

I modified /Applications/IntelliJ IDEA 8.1.app/Contents/Info.plist as follows:


<key>JVMVersion</key>
<string>1.6*</string>
<key>JVMArchs</key>
<string>x86_64</string>

MySQL Replication HOWTO

2009-09-14T09:00:00.003-06:00

I've just published a short video showing how to set up MySQL replication between a single master and slave at http://www.codefutures.com/mysql-replication-howto. This video also demonstrates transaction loss in the case of a master server failure.

When we built dbShards we needed a reliable replication solution so that we could failover in the case of a single node failing but without losing any transactions, which is why we had to build our own reliable replication technology once we had ruled out other replication solutions (including regular MySQL replication and DRBD).

Snow Leopard ate my Java

2009-09-02T08:14:00.002-06:00

Like many Mac users, I upgraded to Snow Leopard on Friday and overall it was a good experience. Some aspects of daily use are definitely 'snappier' now and I do seem to have a good deal more disk space available than before. I had to uninstall some HP software to get my printer/scanner working smoothly but that was the only glitch with peripherals.

I was somewhat surprised to find that I could no longer compile some of my Java projects though. It turns out that Snow Leopard includes Java 1.3 and 1.6 but not 1.5. It actually creates symlinks from the 1.5/1.5.0 directories to the 1.6.0 directory. Unfortunately Java 1.6 is not backwardly compatible with 1.5 (well, the language mostly is, but there are significant differences in some APIs).

If you are impacted by this issue then the good news is that it is very simple to resolve. You simply need to restore the 1.5.0 directory from your backup. This was very simple with time machine, and it was good to have a reason to finally try out a time machine restore.

MySQL Conference 2009

2009-04-24T09:11:00.002-06:00

I've just returned from the MySQL Conference 2009 in Santa Clara where CodeFutures launched the beta of dbShards Cloud Edition. It was great to see such a good turnout at the show (I heard that there were between 1,500 and 2,000 attendees) despite the state of the economy. It was also great to see how many companies are either implementing sharding or considering sharding for their applications. There seems to be much greater awareness of sharding than there was just one year ago and people I met understood that a shared-nothing approach offers the best possible performance and scalability. The one common concern about adopting sharding is that it introduces complexity into the application, but using a packaged solution such as dbShards this is much less of an issue.

CodeFutures @ MySQL Conference 2009

2009-04-14T17:18:00.002-06:00

CodeFutures will be exhibiting at this year's MySQL conference. Visit us on the exhibition floor Tuesday through Thursday next week to see a demonstration of dbShards.

Ruby/Webrick File upload using multipart/form-data

2009-01-25T11:48:00.006-07:00

One of the challenges I'm running into with Ruby is that there isn't the same level of reference documentation as there is in Java. It took me far too long to work out how to implement HTTP multipart file upload in Ruby so I'm posting the code here in the hope that it is useful for others.

The following Ruby source is for a web server with a file upload servlet that accepts multpart/form-data file uploads.

#!/usr/local/bin/ruby
require 'webrick'
require 'stringio'
include WEBrick

s = HTTPServer.new( :Port => 8090 )

class FileUploadServlet < HTTPServlet::AbstractServlet
    def do_POST(req, res)
      filedata= req.query["filename"]

      f = File.open("foo.out", "wb")
      f.syswrite req.filedata
      f.close

      puts "Saved file OK"
  end
end
s.mount("/upload", FileUploadServlet)

trap("INT"){ s.shutdown }
s.start

This is the HTML form to test this servlet with


<form action="http://localhost:8090/upload" method="POST" enctype="multipart/form-data">
File: <input type="file" name="filename"><br/>
<input type="submit" value="Upload">
</form>

FireStorm now supports Ruby

2009-01-21T06:57:00.004-07:00

FireStorm/DAO 4.0.1 includes a Ruby DAO code generator for MySQL (using the mysql-ruby library to interact with the database). To see sample generated Ruby DAO code, take a look at this MySQL-Ruby Tutorial.

The new release is available to download here.

Python first impressions

2009-01-14T11:38:00.004-07:00

Over the past couple of months I have started developing in Python and to a lesser degree so far, Ruby. I have mostly only used strongly typed compiled languages, namely C++ and Java throughout my career and have often resisted doing anything in scripting languages that I felt I could do in Java or C++, mostly because I'm already proficient in those languages and I rarely have too much time for non-essential learning curves.

However, I am now building some tooling for dbShards, our database sharding, replication, and failover solution, using Python. The main reason for choosing Python over Java is that we wanted a lightweight framework that makes it easy to monitor and control operating system processes on remote servers. Python was recommended to me by a friend (thanks, Paul!) and so far it looks like a pretty good choice, although I'm sure Ruby would also have been a good choice for this project.

Both Python and Ruby are object oriented to varying degrees but both allow good object oriented design and implementation making it easy for Java developers to pick up the language. The Python syntax has a couple of things that don't feel too natural, such as constructors being called __init__() and the fact that all non-static methods on a class must have a 'self' parameter rather than having a built in 'this' attribute as in Java. Neither of these is really a problem at all and they soon start to feel natural enough. The only real bugbear I have have with the Python language itself is the indentation syntax - Python purely relies on indentation (tabs and spaces) in the code to determine nesting rather than using curly braces or any other form of begin/end syntax. As a result, I find myself constantly switching between text editors and vi to fix indenting issues at the moment. I will have to spend some time researching IDEs for Python (so far I have been using the IntelliJ plugin, the PyDev plugin for Eclipse, and TextMate and none of them seem great at preserving the indents correctly). If anyone has any recommendations for Python tools please let me know.

Those issues aside, Python is a nice language and easy to learn. Most tasks can be implemented faster than the same tasks using Java, but not to the degree that many report (I have seen many claims of 5-10x productivity gains over Java and these might be true for someone who is not proficient in Java I suppose). The bigger challenge is the Python standard library which is very poorly documented. I spent almost a full day yesterday implementing a web server that supports "multipart/form-data" using the built-in BaseHTTPServer and cgi classes based on the documentation. I think the main issue was that I was using version 3.0 and it appears that this release is perhaps a bit too new to be used for production. When I ported my code back to version 2.5.1 it worked fine.

Python is undergoing some major changes at the moment and version 3.0 was released just a few weeks ago. There are some pretty fundamental changes between version 2.6.x and 3.0 such as "print" changing from being a statement to being a function, that make it impractical, to say the least, to write code that can work with both versions.

I have decided to stick with version 2.5.1 for now and port my code at a later date once 3.0 is stable enough (and once binary distributions are available for the platforms we need to support).

Eclipse "Unable to load default SVN Client"

2009-01-05T11:26:00.002-07:00

After performing a fresh install of Eclipse Ganymede and Subclipse on Fedora 10, I get the error "Unable to load default SVN Client" when trying to import a project. After installing "JavaHL Adapter", "SVNKit" and "JNA Library" whatever those are it now works. Surely these should be flagged as "required" rather than "optional" or "recommended"?

Fedora 10 network configuration is broken

2009-01-05T11:19:00.004-07:00

And the new year gets off to a frustrating start ... I installed Fedora 10 on one of my QA machines this weekend and ran into some problems today when I attempted to change the network settings to use static IP. It turns out that the GUI network configuration tool is broken in Fedora 10 and does not save the netmask value (it seems to write the gateway IP address to the netmask field in the configuration file). The workaround for this issue was simply to edit /etc/sysconfig/network-scripts/ifcfg-eth0 manually and enter the correct netmask value. Unfortunately I wasted half an hour before realising what the problem was. My day can only get better.

Relocation to Colorado

2009-01-04T01:00:00.000-07:00

My blog has been rather quiet lately. The main reason for this is that I relocated from England to Colorado in September with my family as a result of some changes at CodeFutures which I will blog about soon. It's an incredibly exciting time both for CodeFutures and for me personally.

MySQL Replication Lag

2008-09-29T07:07:00.009-06:00

MySQL replication offers a quick and easy solution for having data replicated from a master database to one or more slave databases for failover and/or load balancing of read queries for scalability and is often the first step in a company’s backup and failover strategy.

MySQL replication is completely asynchronous, meaning that the front-end application (often a web site) does not have to wait for replication to happen on each transaction and this keeps the master database operating efficiently.

However, having a completely asynchronous replication solution means that the slave database is never 100% up to date. There is an even more serious issue with MySQL replication because it is single-threaded in nature to ensure transactions are applied to the slave database in the same order that they were applied to the master. This means that if the master database is under load with 100 concurrent transactions for a period of time and the slave is only able to process 1 concurrent transaction due to the single-threaded implementation then the slave database will very quickly fall behind the master database and this could mean thousands or tens of thousands of rows that are not replicated and if the master database fails then these transactions could be lost. I'm sure most companies would find this unacceptable.

I'm curious how developers and DBAs view this issue. It seems unlikely that anyone is entirely comfortable with an unreliable "best effort" replication system for failover although it might be acceptable for scalability if the application is intelligent enough to work around any lag issues optimistically by querying a slave first and then going to the master if the slave is not up to date.

Google have contributed open source code for SemiSyncReplication which provides a partial solution to the problem. Their approach is to have the master sychronously replicate the transaction to the slave mysql instance when the transaction is committed on the master but does not wait for the slave to actually write the transaction to disk, therefore maintaining a sensible balance between performance and reliability.

How do you handle the issue of MySQL replication lag in your organization? Do you use Google SemiSyncReplication or have you found other products/tools to mitigate against replication lag?

Spring DAO

2008-07-20T08:11:00.006-06:00

Spring DAO support has been improved in the latest FireStorm 4.0 beta, which is now available for download. Each Spring DAO method is now annotated as @Transactional and the spring configuration file specifies a DataSourceTransactionManager. By default the DAO finder methods now return typed lists e.g. List<Customer> rather than arrays.

There are some improvements to Spring MVC support as well. FireStorm now generates a single controller per table which extends MultiActionController.

The generated script approach to running shell commands from Java

2008-06-29T05:08:00.012-06:00

Running external process from Java is simple enough using Runtime.exec() but there are some well documented limitations which are covered in detail in an old but still relevant JavaWorld article entitled When Runtime.exec() won't.

I recently needed to launch a MySQL backup from Java but needed to redirect stdout and stderr to file. Unfortunately, Runtime.exec() can only be used to run an executable file and pass parameters in. It does not support more complex operations such as piping the output of one process into another process or even redirecting stdout or stderr. In other words, Runtime.exec() can’t be used as a linux command line replacement.

For example, this works correctly:

Runtime.getRuntime().exec( “/usr/bin/mysqldump mydb --result-file=mydb.dump” );

But this will not work:

Runtime.getRuntime().exec( “/usr/bin/mysqldump mydb --result-file=mydb.dump >stdout.txt 2>stderr.txt” );

This last code fragment runs without error but the stdout.txt and stderr.txt files are not created.

The standard solution to this problem is to write Java code to launch two threads, one to read the output stream from the process and one to read the error stream and then write the output from those streams to disk from within Java, but this seems like a heavyweight solution in this instance. There is also a risk of the subprocess hanging if the Java code does not read the output from the process quickly enough, as outlined in the javadocs:

“Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.”

A simpler solution to allow arbitrary linux command lines to be run from Java is to write the command to a shell script and then execute the shell script. For example:


private static void runCommand(String cmd) throws IOException, InterruptedException {
 
        // generate a script file containg the command to run
        final File scriptFile = new File("/tmp/runcommand.sh");
        PrintWriter w = new PrintWriter(scriptFile);
        w.println( "#!/bin/sh" );
        w.println( cmd  ); 
        w.close(); 
 
        // make the script executable
        Process p = Runtime.getRuntime().exec( "chmod +x " + scriptFile.getAbsolutePath() );
        p.waitFor(); 
 
        // execute the script
        p = Runtime.getRuntime().exec( scriptFile.getAbsolutePath() ); 
        p.waitFor(); 
    }

Using this approach, I can now simply run:

runCommand(“/usr/bin/mysqldump mydb --result-file=mydb.dump >stdout.txt 2>stderr.txt”);

This approach works fine for my requirement, without the overhead of creating additional Java threads. However, this approach is not suitable if the Java application needs to read the output of the process before the process has completed, in which case the standard approach of launching threads to read the output streams should be used.

The generated script approach is very convenient for enabling general purpose linux command line usage from Java.

Skype 4.0 promises high definition full screen video

2008-06-18T15:23:00.002-06:00

I'm looking forward to getting my hands on Skype 4.0. I wonder how well full screen video will really work on standard broadband connections. Sounds a little too good to be true. You can do full screen video with the current Skype version but the picture quality is pretty poor.

Database sharding with Python

2008-06-12T02:15:00.001-06:00

I just read an interesting post over on highscalability.com. There is an early stage open source project called Pyshards that provides database sharding for Python developers. It's interesting to see sharding toolkits emerge for languages other than Java.

I've been working extensively with database sharding for around 9 months now and it's an exciting area of technology that offers a very cost-effective way to implement near-linear database scalability using commodity hardware.

Is Maven the right choice for your project?

2008-06-04T04:39:00.003-06:00

Over the past 6 months or so I have been working with some projects that have a Maven2 build system and other projects that use plain Ant build systems. I love the concepts behind Maven and see tremendous value in standardizing the build process across projects as well as having a better way of managing dependencies on third party jars without storing them in each project's source repository.

However, there are some pitfalls to be aware of before adopting Maven which may or may not be issues depending on your requirements.

First of all, adopting Maven will typically require a Maven server (repository) to be set up and maintained. For some smaller development departments this creates an extra IT support burden that can be avoided with an Ant build system.

Another potential issue is that third party projects might not have good support for Maven. For instance, another Apache project, Axis2, recently released version 1.4 but there is an open support ticket AXIS2-3069 regarding the Maven2 java2wsdl plugin which does not seem to be in a working state yet. The ticket has not been updated since 2007. Sure, the problem can be worked around by calling Ant from within the Maven2 project, but that's adding another layer of complexity compared to a standard Ant project.

Another issue that I have hit is that Maven2 has poor error reporting when it is unable to resolve a dependency. I eventually tracked my issue down to a java keystore / server certificate error but Maven didn't provide any hints that this was the problem, even with debug flags set.

Overall, I'm still a fan of Maven2 and I think the combination of Maven and Ant is extremely powerful but in my recent experience it does add extra cost to the development process.

High Performance MySQL

2008-06-02T04:04:00.002-06:00

I spent some time this morning reading High Performance MySQL by Jeremy Zawodny & Derek Balling (O'Reilly). I've had this book on my shelf for a year or two now and have found it an excellent reference whenever architecting solutions that use MySQL for persistence. The sections on server performance tuning, replication, and backup and recovery are especially useful. This book really is a must read for any developers, DBAs or sysadmins working with MySQL.

FireStorm/DAO on Mac OS X

2008-05-13T03:52:00.000-06:00

The latest FireStorm/DAO 4.0 beta download for Mac OS X now uses a single menu bar at the top of the screen in true Mac style and the toolbar buttons no longer have ugly borders. Sorry we didn't do this sooner but we just started using Apple hardware.

MacBook Pro - Day 1

2008-03-27T02:38:00.003-06:00

As I mentioned a few days ago, I was in the market for a new laptop and was mostly looking at Windows machines until I realised that there was no good reason why I shouldn't buy a MacBook Pro instead. I decided to take the plunge and I ordered the 15" model, which arrived yesterday.

I've been developing on Intel machines since 1989, starting out with DOS and then working my way through Windows 3.1, 2000, NT, XP and Vista so to switch to a different OS is a bit of a leap of faith. From time to time I had considered installing linux on a laptop but frankly it is too time consuming to get things working and I have precious little time as it is.

The main attraction of moving to Mac OS X is that it is linux based and I can now use many of the same commands on my development machine as I use on the environments I deploy to. I also like the fact that the hardware and software have been produced by the same company and should therefore provide a stable and optimized platform.

The install experience was easily the best I have experienced on any computer. I was able to check out a couple of Java project from subversion and get them building with minimal fuss. The MacBook Pro built one of the projects about 25% faster than my Windows Vista desktop machine which on paper has a better spec, so Mac OS X does appear to be more efficient.

I love the fact that there is a DVI port built in and this gives a crisp image when connected to my 24" Samsung monitor. The MagSafe power adaptor was a bit of an experience. I'm not ashamed to admit that I had to search on Google to find out how I was supposed to disconnect it.

I've installed vmware fusion (an impressive piece of software) and I have a Windows XP and a Fedora 8 virtual machine, primarily for running unit tests on those platforms.

It will take me a couple of days to get used to the keyboard but as this will be my only machine on a trip to Colorado next week I'll be forced to get up to speed quickly.

I'm extremely impressed so far!

Sony Vaio or MacBook Pro?

2008-03-20T03:21:00.003-06:00

I have the opportunity to buy a new laptop and I have a budget of £1,500 (plus VAT). My main requirements are decent processing power (2.4GHz Core 2 Duo + 4GB RAM) and a good quality high resolution display (ideally WUXGA 1920x1200) and a keyboard that it suited to development work (so functions like 'home' and 'end' can't involve complex multi-key actions involving several keys at the same time).

My initial shortlist included the Lenovo Thinkpad T61p, the Dell Precision M6300, and the Sony Vaio AR61ZU. I was pretty much settled on the Sony although I was concerned that this might be a bit larger than I really wanted due to the 17" display. I've now realised that the MacBook Pro is also a contender so I now have to choose between that and the Sony Vaio (which is pretty much the Apple of the PC world).

Apart the basics of email and web, I'll be using the machine for Java development work 90% of the time and the tools I use are available on all platforms. The only Microsoft specific tools I need to use are Word and Excel. They are both available on the Mac, at a price. I guess there's always the option of using OpenOffice but I don't know how well that really works, especially with Office 2007 files.

The MacBook Pro is probably the better choice for development work given that the OS is linux based (most of the development work I do is targeted at unix or linux production environments).

It's a tough call. Suggestions anyone?

More on Hibernate

2008-01-27T03:22:00.000-07:00

I got some flack a few months back when I posted an article about poor scalability when using Hibernate for a proof of concept I was working on at the time. For that particular project it seems that Hibernate probably wasn't the best fit. However, I've now had the opportunity to use Hibernate on another project and my experience has been much better this time. First of all, the productivity benefits of using Hibernate rather than writing Spring JDBC code are quite impressive. I was able to get a prototype web application up and running in just a couple of hours with Hibernate because I really did only have to focus on the business logic rather than writing low level code. Scalability isn't at all critical for this application since it will not have a huge number of users but I ran some tests anyway to compare one of the transactions with a hand-coded JDBC equivalent. Here are the results:

It's true - Hibernate does add some overhead to the transaction but it's a price I'm happy to pay for the time saved in development. I expect there are other transactions in the system where Hibernate will be faster than JDBC due to caching.

I still stand by my previous conclusion, which is that it is always worth benchmarking products/technologies against real use cases to make sure they are suitable choices. One size does not fit all.

Sun to buy MySQL for $1 billion

2008-01-16T16:27:00.000-07:00

I just heard that Sun are acquiring MySQL for around $1 billion. The press release is here.

Sending extended character sets with OpenSMPP

2008-01-04T06:35:00.000-07:00

There's astonishingly little documentation on the web about sending international characters in SMS messages using OpenSMPP (which is based on the Logica SMPP API). After a bit of research and some experimentation I've established that the following steps are required:

1. data_coding must be set to 0x08 (instructs the SMSC to use UCS-2 encoding - see section 5.2.19 in the SMPP protocol specification for other data codings)
2. When setting the message content, it must be encoded with UTF16

Here's some partial source code to demonstrate the point:


 SubmitSM request = new SubmitSM();
 request.setDataCoding( 0x08 ); // UCS-2 
 request.setShortMessage( messageText, Data.ENC_UTF16 );

It's worth noting that the maximum message length is reduced from 160 characters to 70 characters when using UCS-2.

MySQL SemiSyncReplication

2007-12-31T02:36:00.001-07:00

One of the projects I'm involved in at the moment has required some research into strategies for clustering MySQL for HA. One of the interesting ideas that we came across is some code from Google that adds semi-synchronous replication to MySQL's built in replication feature. Standard MySQL replication is asynchronous, meaning that transactions are committed on the master database and at some point after that (could be milliseconds, could be minutes or longer) the slave databases will receive the transactions and replicate them. This is great for performance but if the master database dies then transactions are likely to be lost due to the lag between the master and slaves. One of Google's contribution is to make the master database wait until the transaction has been received by one or more slave databases before acknowledging to the client the transaction has been committed. This doesn't 100% guarantee that transactions won't be lost (the slave could die before the transaction is actually written to disk) but it dramatically reduces the likelihood of transactions being lost. Of course, the price to pay is that writes to the master will be slower but thats pretty much always the trade-off between performance and reliability.

Here's a link to the Google MySQL Tools project.

http://code.google.com/p/google-mysql-tools/w/list