Scalatra Rules

If you’re doing Scala development and want to save yourself a ton of time and headache, seriously check out Scalatra. I’ve worked quite a bit with the Play framework and worked against Spray, and I have to say that Scalatra is just a joy to use. I might name my kid Scalatra. Scalatra Legler. It rolls right off the tongue. That’s how impressed with it I am.

I’m not going to knock Play too much because it is a great piece of software. It’s just really heavy if all you need is web services though, and I’m not crazy about the functional testing framework (I’m looking at you FakeApplication, you crashy bastard). That’s beside the point though. I’m building an app that makes heavy use of web services and websockets, but the entire front end is javascript. I don’t need Play’s templating etc, so it’s just overkill. I researched micro frameworks and out of several, I decided to play around with Spray and Scalatra. Actually, I shouldn’t say that. I decided on Spray because it is so tightly integrated with Akka, which I use extensively, and so began the trouble.

The Spray toolkit is nicely decoupled, so you can use just the part you need, which I like. I love their caching module, in fact, I use it with Scalatra. I found their routing really cumbersome though, and I hate how implicit everything is. There is a lot of magic going on, which I don’t like. I’m building a REST service. If I want to send a response code with a json payload, that should be trivial. I shouldn’t have to look that up. I certainly should have to spend hours looking it up and learn an entire new vernacular of Spray idiomatic concepts to express what I am trying to do. Yet hours into messing with it, I found myself in the bowels of various google groups finding out that I have to write customer marshalers to do it? (again, we can’t agree to call them serializers? We have to come up with different terms?) Fucking annoying. The problem that put me over the edge was trying to return a 204 Status message if there were not results to display. This is a really really really common thing to do, and for the life of me, I could not get Spray to do it. This should be a one liner. I also wanted to throw 400 status codes when url parameters were missing with details as to what was missing in Json. Again, totally normal, should be a one liner, and a total fucking nightmare in Spray. I am sure that once you learn the idiosyncrasies of Spray, it’s a delight to use; however, I have shit to do and it was driving me crazy. Enter Scalatra.

At the point I went to Scalatra, I was completely disgusted with the Scala community’s acceptance of stupid complexity. This happens regularly for me. The people are great, but the idea that this complexity is acceptable is irritating. The Scala community at large needs to step back and start thinking of best practices for the language. It seems like the people with the loudest voices in the community have raging boners for the most complex abstract mathy ways to do everything. It’s exhausting. The thing about Scala is though, that if you ignore those people, the language is awesome. You don’t have to go down the pure functional hole to get performance out of it. I write Scala using the Zen of Python as my guide. It works great, and it makes me not feel guilty about ignoring the people running things. In any case, Scalatra is sweet relief from the abstract implicit horseshit going on in the Scala community. They’re my kind of people.

Let’s get back on task here and use an example to do it! For example, let’s say you have a parameter called name in your url, from which you’d like to get a value, probably a name if you’re into obviousness the way I am. Let’s say you want to throw a 400 code if it is missing and a json response explaining what happend. There are lots of ways to do this, but an easy one for me was something like this: Create a case class called HaltReason that contains a string called error. You’ll use this to deliver the message to your users of what the problem is. In your Servlet, extend ScalatraServlet with NativeJsonSupport. You do this to convert your HaltReason object into json. Then get your value with this line of code:

val name = params.getOrElse("name", halt(400, HaltReason("missing name parameter")))

Boom, that’s it. It either gets it, or it throws a useful explicitly defined error with an explicitly defined message to your user. Did I just blow your mind? If I hadn’t written that example and explanation, I’ll bet it would have pissed me off. I need examples man. I can’t conceptualize things like that. To remedy that, here is an example of the code in context.

Note: I didn’t compile this, so there may be syntax problems, but you get the gist. I also added a method that will always return a 204 response, just for awesomeness’ sake. You can imagine writing some code that results in no result, and then you can probably imagine explicity returning a response that says so. Imagining is fun.

import org.json4s.{ DefaultFormats, Formats }
import org.scalatra._
import org.scalatra.json._
 
case class HaltReason(error: String)
case class MyDumbResponse(name: String)
 
class ApiStoreServlet extends ScalatraServlet with NativeJsonSupport {
 implicit val jsonFormats: Formats = DefaultFormats
 
 // I put all of my routes here, this may not be totally normal. Check out the Scalatra website for real examples.
 before() { contentType = formats("json") }
 get("/getName") { doSomething(params) }
 get("/throw204") { doSomethingElse(params) }
 
 def doSomething(params: Params): MyDumbResponse = {
  val name = params.getOrElse("name", halt(400, HaltReason("missing name parameter")))
  MyDumbResponse(name)
 }
 
 def doSomethingElse(params: Params): MyDumbResponse = {
  halt(204) // This is just an example. I didn't even make it try to do something other than throw a 204 response.
 }
}

Better? If not, check out scalatra.org and build an example project and try it out. Try to do the same thing in Spray. Actually, if you do it in Spray and get it working, send me the code. I gave up on it and I’d love to see it. My point is this: Scalatra makes it easy to do exactly what I wanted, and that is my favorite thing when it comes to software.

So what about Akka integration and all of that? What about the fact that it runs on old and busted Servlets. Don’t worry about it. The akka integration is perfect. You spin up an Actor system behind Scalatra, extend your Servlet with Future Support, and you can distribute and cluster until your heart is content. Worried about thread exhaustion because of the Servlets? Don’t worry, Scalatra works with Servlet 3.0+ containers, making that not a problem. It’s already integrated with Atmosphere, build some asychronous beast and make a billion dollars. Scalatra makes this stuff fun. I’m sure Spray is awesome too, but I just got frustrated with the complexity of it. I want my framework out of my way so I can do the things I want to do, and Spray just wanted my attention constantly. Play is great, but it’s heavy. Both Play and Scalatra are well documented and straighforward. Scalatra is just lighter, which I wanted and I like. There are others out there that I didn’t try as well, and I have no idea how well they work. Try them all if you like. I just really liked what I found with Scalatra. If your time means anything to you, I imagine you’ll find Scalatra to be a pretty valuable piece of software to use.

Ch Ch Changes…

I’m working on a new project now, which has been gobbling up my time.  Consequently I have moved DenverDesignStudio.com over to WordPress, so that I can focus my time more on the project and less on developing yet another blog platform.  If you haven’t checked it out, Ashley is doing some incredible work.  What was originally intended to be wedding calligraphy has become a whole bunch of different things.  She’s doing wedding calligraphy of course, but she’s also doing maps, family trees, Christmas cards… It’s pretty awesome.  Surprisingly, she’s had as many international customers as she has local customers.  It’s pretty awesome that a little shop in Denver can get clients from all over the world.  The site is running well, and there has been an unexpected extra benefit to moving over to WordPress.  Because it isn’t running on the JVM anymore, the memory requirements have suddenly become very modest.  That means I can move to a cheaper VPS solution.

I moved this site, and all of the others that run on my servers, to the cheapest digitalocean VPS service, and they are running really well on ubuntu/nginx/php/mysql.  $80/month vs. $5/month.  Not bad.  Technically, their response time is about 20ms slower than my linode server; however, that’s most likely due to the new servers being in NY instead of in TX.  The load times appear to be about the same.

I haven’t had to deal with their support, but it was really easy to get things up and running and it took me about 4 hours to configure the server from scratch and move 13 sites over to it. I know that isn’t fast, but I didn’t have any of it automated because I hadn’t really planned on moving it, so lay off me.  Overall, I am pretty stoked about digitalocean and their pricing.  I hope they’re making money and can keep up what they’re doing.  My VM will probably fall over if any of the sites get slammed, but for $5/month, I’m willing to take the risk.  I’ll upgrade if I need to (and it looks like it would be pretty easy to do).

Formatting

I’m a sucker for formatting.  I follow the python PEP-8 religiously when I write python.  The Scala style guide isn’t as dogmatic though, which isn’t good for someone like me.  I like having some rules.  Anyway, I’ve been writing decent Scala code; however, I’ve not been confident in my formatting choices.  I’ve been getting more confidence through a Scala Coursera class I’ve been taking, but even with that, I just wasn’t positive I’ve been doing it correctly.  That has now changed.

I was just recently turned onto scalariform by a coworker.  It basically checks/fixes your formatting on compile.  It’s completely configurable, but so far I’ve just left it alone.  I set it up to check any code I compile with sbt.  Between sbt, scaliform, and intellij, I’m feeling a lot better about how my codes looks.

More with Max/MSP, Ableton, and now Python!

It’s pretty neat that so much of what I’ve been interested in, as far as technology and music go, are all kind of congealing into one big mess.  This weekend I started exploring the capabilities of this software call Kapture.  It basically does exactly what my MIDI mapper does, only it captures all configurable parameters of the Live set instead of just the parameters I had the foresight to write code to capture.  It also saves them to the set instead of a separate xml file.  It’s pretty badass.  It basically lets me snapshot my entire setup, save those snapshots, and then recall them whenever I want.  The only part I was missing was a way to get it working with my FBV controller in a way I liked.   Bring on the python!

I wrote a python script last night for the FBV Shortboard MKII that lets me control live with my FBV.  It is based very heavily on the work of Hanz Petrov who has done so much work documenting the LiveAPI that the code is pretty trivial to write now.  My code draws a red box around the tracks I am working with exactly like the APC40 does, and basically makes the FBV a physical extension of what is going on in Ableton Live.  This basically gives me the ability to create snapshots of all of my virtual instruments, mixer settings, etc. with Kapture, and then I can stomp on a pedal to switch between them.  My hands are free to play instruments, and my feet control Ableton.  Pro tools may be the industry standard, but this flexibility and stability makes it feel like a dinosaur.  I can do almost anything I can think of with this software.  It’s incredible.  Python is kind of a glue language, and that is exactly how I used it.  I glued some hardware to a neat Max/MSP app and have something better than what I was trying to write earlier.  I learned a lot writing my old setup though so it isn’t a total wash.

This functionality has obvious live music implications, but it turns out that it is so efficient that I can probably use it when recording and mixing too.  I barely have to touch my laptop which is awesome.  I am now working on getting my Maschine integrated with everything and I should have a really flexible setup with a pretty minimal effort.

I am cleaning up the code now and I am going to try it out over the next few weeks.  I’ll post the code if anyone wants it, but for now I am going to play with it a bit first to make sure it all works as expected.

More Ableton and MAX

I sat down to write this post awhile ago and life kind of happened so I am finishing it now.  A few months ago I sat down to finally write a MIDI mapper for my line6 FBV Shortboard Mk II so that I could control Ableton Live with a foot pedal and not have to worry about using my APC40 for live performance.  I wanted to do it because I’m not a DJ and rarely have my hands free when I am performing.  I’m usually playing bass or keys, and I need both hands.  Trying to work a device with my hands while I am using them to play a different instrument is a wicked pain in the ass. Ari also pointed out that my setup was kind of byzantine and simplifying it might be a good move.  I agree.  My solution was to take the FBV and write a tool in Max For Live that would take the CC messages that the FBV sends and turn them into midi note messages.  I initially wanted the bank buttons to set banks and then based on that number send different midi notes so that instead of being limited to 13 midi buttons I would have x banks of 9 buttons.

My initial implementation worked exactly as designed, but when I tried setting up all of my band’s songs in a single live set (so I could switch between songs seamlessly), I quickly realized that midi was not going to be the solution.  I had not way to track what state the tracks were in from the song before which meant that I had to put every song in the order that I intended to perform them beforehand.  That sucks because it doesn’t allow me to change the set list on a whim.  I also ran into a limitation of live where I can only assign one midi value to a parameter.  This meant that if button a was controlling parameter a of a plugin or whatever, button b could not control that parameter at all.  This meant that instead of being able to hit a button and have it do a few things, and then hit another button and have it do a few different things and undo one of the previous things was not going to work.  The whole thing felt very fragile.  I decided it was time to use Max For Live and it’s Ableton API to set parameters and just take as much midi out of it as I could.

My second implementation used a bunch of set commands to just force set parameters.  I used the param save function to save the settings for each button press into xml.  That worked really well but I later found out that the looper built into live basically requires midi to be used.  It doesn’t expose anything I need to the live API.  I also found out that if you use the set command to set a parameter, it does it as if you clicked the button yourself and there is a 10ms delay or so.  It also has the side effect of creating an undo event for every single thing the pedal does.  That probably isn’t a big deal but I found out that it isn’t really that hard to use the remote~ function to send messages to live and anything sent over a remote is realtime and and doesn’t create an undo event.  Basically the goal at the point was to use remote~ for everything I could and use regular set commands for everything else.  Below are some examples of some of the things I’ve written that are part of the entire tool that lets me basically DJ my own music with my feet while I play.

Here is an example of setting a return track volume with remote and with a set command.

 

 

 

 

 

 

 

<pre><code>———-begin_max5_patcher———-809.3ocwW97jZBCEG+L9WQFl83VKgeI1Y1CsS6sdpSusSGmH7TSKj3PhVa2Y6e6kj.JKhUTVrGTjGgju7488Ed9zHK647cfvF8NziHKqmFYYoCoBXUdtkcFYWbJQnGlMC9Ie92su2bIIrSpCKnK+SUvEblTP+Mnt.1crSYX1lLJKEj54AeHHeirYTZhdRKVn23EVeZYjL8zZ+9bJIs5JlYP9q0f4IQoFVwkQeqb.qIx3UT1xY4PrzLlPmBgglhCTG77Te6VDB8M0c77nQputueLIktEFmCYbIbQrwsU13zBalbd1b5Gce8iN1cxgm8x6QP1BIyJVghaYFQJyoy2HM1Dq8Hvxd1ZHWPERfEaddzw0r6RIXFHDjkvQHrP8qPJNNS.RjLmD+CA5NLJitCxmk.aow.R.rDAxo5lSoLHlugIqyxqm6s4IitBO4+zMhCz4jnI5CXiezyc.7iJh1+BUuighuy0.k6K+zA3TVqFDLD0pzEn6ntnGd.4fjq.VwYX6gv6fcd0MOl54IFJgclND7QhVfl2eeiaK9lqgGKR4Dox3LmvV1A3DgKgSz3fd.mFaS85ZLlLTap3a7GZeANZ.pavp5FrotwYbT.BREPwuFjFBvdCT8SjwhLL6uHQyQzAo94Z5ORW0TT8PUuj7rjIbpgLg8q3IlmkAEKXSz7U0a0QelyKZm.i9DiLOEd6GoB0w9SrVZaJH3pZaZh1XDZNfivG01jZ1VPp0KzEBnbRBkuLmuYckJ1Rx2KOcyOkT5KGMxKnVZcNTzwjjHobV8bcjw9WsKQ0dEGyOW+dUm45bhkntvdQwuDxpVipX6MC6IU5lp+HygYLw3hR1G+Uz1pREs1hPz0XOOQRIx6PRw276v1SJ3om1Ti9.OM4rN6xDiu+QKyQIlZXTOHcO2M9aj50QE+krUv2jGWYW12PT8bFHjT1905wC65WaPqnII.qN.ynIq4E6mUphSjp6pnTV7yqoFJenEUXGzTCgeKjD9bXZxKFzPKoFNk10j+sMyoWtNwoanlb5fl7ts4NutXw8tsbxqKaOog4sSSMrJmfS+G73mUSg8PSEm77n+huFKWB———–end_max5_patcher———–</code></pre>

Here is an example of setting tempo that takes 3 seconds (if you’re using a 44100 sample rate ) to get from where it is to where you want it (for smoother tempo transitions if you want to link songs together and have a drummer playing to the click track… which I do).

 

 

 

 

<pre><code>———-begin_max5_patcher———-548.3ocuUF0bhBCDG+Y7SQl7rmCATuZeq2Wia5vDgUM2QRXHKddWm1O6WR.rTEKZq0GHPV1jc++a2.OMJftTuCLTx8jeRBBdZTPf2jyPPy7.pjuKMma7tQUvezK+Ecb8qPXG5MijUjksVWoUnQ7Ov8FVzjvFypJoPkCneiXuZTWgsViZrJx76pMRei0cWUboeWoOTJ34suodCv+V.0JgtJWyQ5XBcIWslRdrwuBNltQnVmTBoXsqSmYSOx28ir42MY1XRT3jPxitU77nQtgweNzjK1BSJAoFgW5kPg8SnndIT3wDZ5rSiHxOz4YzgzO6NWNPlundhS+MKwv2BYI1nXWQBGwRwxJrtiIXOFBnIEPoQXPPkVqIucO+tlTzJfMD2SIF.IRtMfkIXIO82DoXm84LXqHEHFsUjHHKzs6StPAo5JE1krmSUn+9z3dpBwmYU3ntUWip+58ZTYKpqTwMsp9BVzhqaqZIWVXjZMt4EajhhCCatcI8sw8RL1wDKd9GkXFwZk83+vGraw0zNc1WObYSiK5DM6rIyrudxD5uEy9TfwqMZeZM7b0Zz7om9jwv5ndLZ+39UjpkRn97Nk1QZdG7eO3f+64Cgy9a0qQWUl1lPM+Ph7ZXx.CJTbTnUc7wU.63zFQVFn5hEoHqPKTXSNbB3eIoDafTx88oaXJ4KqCxI1MMm7HXPNM6MN8UmSGzobh1o42TNcP3dGN8QyI6jmG8ebm4y0N———–end_max5_patcher———–</code></pre>

Here’s a little event timing manager that lets me make my pedal presses take effect on the 1 via quantization:

<pre><code>———-begin_max5_patcher———-1951.3oc4b0zbaaCD8r7uBLZxsl3AeQBxNMGZu068VmLZnjfrYsDIKIThcxj+6EePJKKKIBIxEId53wlFTjhKd3sucAVH8salLcd4ixloneE82nIS91MSlXOk4DSZaOY5lrGWrNqwdYSKjeob9+L88tWRIeTYOc9Jz6xonO9QDFotWVnaQPx0MRT4VE0zp6VVUVnZx+pzbaD5s31SWrcSdwZox9TnOeR8se3YyWZejZy3Ch8eWKx1XeWm9604Yq6dE2af5oJoqaNc560+h9T6KWkoVbedwcypkKTtqfGosJDI1cHgXNPw2hQexbKe+laL+48dhW1N0zi0IwGsSRdcmL5jck7B0Y6ITrv1GX1tBc2e2cOKJ2rQV3FBmdE8tdYCjPxFHD.oCoNzCGOH5vFYSS1cxWgXU0kUxZ0Sn7lYUqydRaBCGhNBWJ9ZPnyRwXFSpCjHDmOyUhNmfNsN+yxaKm2Hq+rrFDlCGBhSKzHbPiH8YhS68zj8Y4xY5mu9VlkoT04y2pbxwS1gPSlNSyLZxaTxhE1mM1ddKzNl.rtGbOx7eyZdVwxGTlbTTl8ZTldsnbeHMKZORXZ5f7POADoPyQyGNpbLUqq.UlmYzGZO5AGjXQFFePHSWvhCgl+597Fj8YKqQprGjMFQejN3j7N8YZz2CRUhxUnrhknkRs7Vio01BU9Z6kpeDJzFYVy1Zo8hnbcZD4KdnAMWtpTeR8kqeeVZtuaQn+Tgz5n52WMFs9Iz+tMS+ueU+byJdBo8gTkEnpZykzXt5U1GhpNawCZ8UTiprpRtz7dZFVruXqtroezXNSc416tW2EZT5250OcaWmdcdgbQ4VWHyjASGvulNv5mNbjwYijql56TewNsXwX6AjeBO.7.8.Xryn+h9ix0KOaFP93G3jFXNsXF61nwOJUUdk7RfGuiYyXjqEd7.SRr1GPXhQ0LGFNC8pAkNwy9xdtMsYA0BPQ3AAPmNwOYkVXSOso2Qfg8PDfvdZmejHB6.I2blFYzAqm.AUOLvSXLr.H7IFT7woFmxGD9Dd2KbJztWwVfIlMFdWmAeVAD9jLT7Y05xryiPbrkBEicJzz2XJzXAzTHpK1E3.DTbnX34Pznn8byhigAhVAFDc0Q4cPieXjiFEwaUjda4mQSRf1OqUHp6PJD.jcMCIneqaAjw1e5eQCGL7It5Hc8ttDsg3RcSHKlNnEG6T4HAb9QTQBH4G0IL0hQBHxOR9XUM5cqHneAwSvCm9bT3gCJ73BsIDPrtpFWtU64xYZrmC2Jnb3nf4v0hZQhVOtTHfMGqRqU8ADADNEgx.gSEue1R5vbPnFQ.RDR.Ih3NDQfPCx5jQ2yIyqRggGXAMnhXfipE.eLNFq8wFEcni4kQfYY05xlzcPvATBhZfmbfV2HLjhPsnSp.DQHNPHBEl0gsMsZ2xMFgAYkz1Uf4l76JxTaqkyVJKJ03PlprFFQ6zDHwK1K2mFwgtlyCVhNEtIdzAQt5cPo6Eq+MZsmwCr1yLLdPnceHdTx9jxTP1GDaKpLkvzNy3aAAknozAVIt1C8unKt3.71xGv.YeiTlszsDGCsNsGScies0s+rUMvgGTQap5fsUFb+LbRTzqQlzAurlm8nGvGyMEmHZJHoWrKV5hs00xB0rlRsUnx2.S8eoIvj2QGX4VZXBm+FMNZBCr3ncPTmiXz+6iiRS3fFGs00skTBSbzunmGNPKEez0BNq0jiKRhKskYByp3fHWXJFWP814.NuRtayZPvoCRLaD101L10uusamYCXaa6Qo6Ee8Y43TSckaCftmyflNrsgGSLs2sNk3R6A1KvtIBO3Ccg8YXN+K6VMkaqWzAq6RPGQ28fVJaTlovmWVr2UYWZKzyly84K0S1e+d5l7kUkZZXqYbBmDespCedG2prEyMfVkd.TKi0iMYFluDahRbkPOsUtw8YCXuFIt8hbD2bY1FiP2.26.dPQVq1VuFkHn1juVDIbnDwmgN69OKjicl00sWqxV+f.ZUNcsd8VOv1g1p3955ENRkYx08hS1jzCHNkF6CkxLM1PRo7YzKMN7.EqWVNN7FUeoTXWL+.ZT9vnBKI2mTbBrnoGRA7fpNQ7QbJvg6ndkZJ6xvINwleGO1s.EbwtVC1dw9fglZGEvwUW.k9C6vBqUcflz4vp.x3Hburp.GMLIxqjQ4AlW4UxnGX6vaUTurpvlL5gojehzQog1p7BqD7.OchTuFASBLuJwqXQze.VEwKqJfSdF6UtDhPmMAw6IOGRrJwKrJ4BywgZ+PEGGYKLHiX+30XaMJ1K1K6Eew4jE61Gmz3maMJZb86gDX0DA2K0jPu7RwPL1NJV0n6gLJQz6mWIBLV4UrSwOBqh3kUEv46RYd4CF3YffEPLBRIo6ouFmNVxqNyczGZoLlK7kaOFiIN6ULBgCR7dc8CXR.TeRX5vZRDFq5msZfbH0+TLtz.aU9HwcHhB+xY3S.c62QIAMzY5OgVk8KxFO7A4AtZsbuwpPZUdoLvD+7UznKDnHQ1PkT22AHb2dG4ksHs6HFaLJBdvgn7a+XvBbATHfrgLbfV6mwA2dv3EMF8MjgcqO0eziKDbcbj3D2WWaO25EFqtw2u4+.JDKslB———–end_max5_patcher———–</code></pre>

With all of this working I am able to control Ableton with my feet while playing piano or bass or whatever.  I never have to touch the computer which kicks ass.  So far I’ve not found anything I can’t do with this software.  It really is a pretty brilliant piece of kit.  The sad thing is that I think I like writing software more than I like writing music.  I’m certainly better at it.  Anyway…