Why isn’t Autoland working?

This question comes up enough that I figured a quick blog post/status update would be helpful.

What does Autoland do?

Poll individual Bugzilla bugs for an autoland token (currently using whiteboard tags, future Autoland has an extension & webservice that makes polling the entirety of bugzilla no longer needed). When an autoland request is found, the serviced does an automated landing to try for you of all non-obsolete patches attached to the bug if they can be landed cleanly on tip of mozilla-central and either the patch author or a feedback provider have appropriate hg permissions, otherwise it reports back to the bug what the issues were. Upon completion of the try run, a comment is left in the bug stating the results and if a final repo destination (or destinations) was specified (the hg permissions must match up between requester/reviewer and the destination repo(s)) the service can continue on to autolanding the patch(es) to the destination repo.  A comment would be left on the bug when the push to final destination(s) is done.  There would be no reporting back of final build results, that would be handed back over to human eyes on TBPL.

What is Autoland’s status now?

In April of 2012, right before Marc’s second internship ended, we launched a very experimental public-facing version of Autoland and announced it a bit so we could get more people testing it.  This had varying degrees of success.  We got more bugs ironed out but also discovered that Autoland’s daemons for hgpushing and bugzilla polling tend to fall over a bit too often.  When we moved Autoland off it’s staging VM to a more permanent home we lost the status page that would tell us (and developers) what the modules were doing and that really made the workings of Autoland quite opaque.  With Marc leaving at the end of April and my switch over to help with Release Management in Feb 2012  I had kept meeting regularly with Marc and driving the project to completion as much as possible but hadn’t been able to pull my weight on coding for the last 3 months of our time together. This left us with an Autoland that stopped working and no one available to continue to massage it into being the robust system we needed it to be.  I took mention of Autoland out of our trychooser page, try server docs, and have generally tried to downplay it’s existence while still keeping a plan on the back burner for how we will resurrect it as soon as there is some time.

What does Autoland need to be publicly usable again?

  • A status page that can show what modules are running or down, display what’s in the queue, and give a quick visual to users if Autoland is up or down as a result.
  • Nagios alerts on the Autoland modules that let me (and other people interested in helping to maintain Autoland) know when things fall over.
  • At least one person, if not several, who can access the Autoland master VM and ‘kick’ it as needed

This is what’s needed for a short term solution. I know that we have some bugs with our hg pusher module as well as some trickiness in our message queue that, once fixed, would make the overall system more robust.  We need people using the system to be able to catch more of those bugs though, so in the meantime having as close to on-click restoring of the system would be a huge win here.

What does Autoland need to be truly ‘production’ ready?

  • Security review on the code and the BMO extension so that we can move away from whiteboard tags and let people use the BMO extension instead — this gives much cleaner input to Autoland of what is being requested.
  • More VMs to run hgpusher modules on so that Autoland can handle a larger load. Each VM can run 2 hgpushers max so we’d want to be able to grow our pushing farm as the usage of the system increases.
  • Being able to push to repos other than Try.

There is no clear plan to be able to get the system beyond Try landings, but I see automated try landings as still being a huge help so I’d be super happy just to see that part get back to a working state.  This project is no longer a RelEng priority now that I’ve permanently moved to Release Management and Marc has gone on to other internships and more schooling. I can’t promise anything time-wise, but I wanted to provide some clarity into what is needed and put out the “patches welcome” call.  I see Autoland as being a great option for a community-managed project and I want to keep working on it when time permits. If you are looking to become a Mozilla contributor and are interested in automation and web APIs – this might be a good starter project for you. Please get in touch.

 

 

Isn’t “hack” a bad word?

This past weekend I led another soft circuits 101 workshop as a Mozilla Rep at a women’s music festival near SF called Fabulosa. I had one hour to teach people really basic electricity, circuits, and how to ‘hack’ their clothes/sculptures/lives with a 3V battery and some LEDs.

The reason I love to do this workshop is because I find it gives participants a physical representation of the hacker spirit Mozilla aims to embody for the web.  Learning soft circuits is just the tip of the iceberg and I always stress that the web has much more info for them to continue exploring, learning, being curious about how to create and modify technology in their lives.  In one hour, I just show them how to make light.

This workshop was smaller than the ~80 girls who came through at Dare 2B Digital.  The festival setting meant there were more conflicts of interest so I had 6 participants instead of the 20-25 I had planned for. The 6 participants were all very enthusiastic though, and we started off with a go-around to hear why people were interested in learning soft circuitry.  One person was hoping to learn how to light up her clay sculptures, another wanted to make art for Burning Man, and a few had costume ideas in mind.

I spent the first 10-15 minutes explaining electricity, how a circuit works, and what kind of circuit they would be creating with their 3V batteries and LEDs – their circuits would be made using conductive thread sewn into felt (or some other material if they brought it).  With only 40 minutes left, we got to work – everyone started in on their first circuit.

While we were casually chatting during the building time, one woman said “I thought ‘hack’ was a bad word” (I had written a large “HACK” on the whiteboard to inspire).  I’m so glad she brought this up and we had a chance to discuss the very reason for workshops like these, and for Mozilla. I explained to her that while it might once have been a vilified term, it has now been largely reclaimed as people work to make sure that they have full ownership of the things they buy, or make.  Encouraging people to open their minds up to the potential of hacking their lives – whether on the web or in the physical world – always feels great.  I was happy that even in this small gathering, we got to discuss this very key issue for technology going into the future, and that there are now 6 new hackers in the world.

App Marketplace Ratings

Woot!  Our recently release re-vamp of Firefox for Android is climbing the Top Free chart over on the Google Play Store (as it should, it’s frickin’ awesome).  We’ve gone from #96 to #81 in the past 3 days and I have no doubt we will continue to climb as we gain users and get a chance to impress them with the Native UI which is responsive, beautiful, and support Flash.  Our rating in the store is also slowly climbing, but that’s going to be a much harder slog because our current rating still reflects the total collected in the entire life of this product being on the store.  We can’t remove ratings from our previous Firefox for Android and so even though we’ve had 5,000 5-star reviews in the first 10 days of the re-written version being online, our average rating is a 3.7. The only way to get a fresh start would have been to put up a ‘new’ product and call it something else and I’m sure you can understand that Firefox can’t go by any other name.

I spend a lot of time thinking about the Google Play store and how it could be better because I interact with its administrative backend regularly, uploading new builds of the mobile products release after release.  I was just sitting through a presentation by Dees at ReMo Camp 2012 in Berlin and he was sharing with us the strategies behind the Mozilla Marketplace. This will be our open-source contribution to mobile/desktop app distribution and seeing the mockups started me thinking about how we could improve the rating system and not just repeat what Google and Apple do with user feedback.

I’d like to see the following:

  1. User leaves rating, gives stars + writes text feedback
  2. App developer can select reviews to flag as ‘bug report/feedback’ which requires them to write text that will be presented to the user.  The developer can write a message either letting the reporter know that a bug is on file now for the issue or provide help with the issues/questions raised by the user.
  3. User gets a notification when the review is flagged and that there is a response ready for them. They can check out the bug report that got filed as a result of their feedback and perhaps they will cc themselves to know when it gets fixed or they might get a chance to try out the suggested solutions from the dev to deal with issues or questions they raised in their original review.
  4. User, now that they have gotten feedback, gets prompted to revise their review.
  5. Repeat 1-4 as needed

This would be beneficial for many reasons:

  • Users get to be a part of helping improve the product
  • Users get support from the developer without needing a different forum or login
  • Users get visibility into software development process, awareness of upcoming features & improvements, and they become participants in open source community
  • App developers have a channel to communicate with users about upcoming dev plans, feature requests, and bug tracking
  • App developers get a collected feedback average that is more accurate and representative
  • App developers have a channel to communicate with users about upcoming dev plans, feature requests, and bug tracking

I’m used to Mozilla’s collaborative environment, the values of open source, and I’m accustomed to getting feedback in our open bug tracker, Bugzilla.  There are so many companies whose products I use who do not have public bug trackers and this causes me a lot of frustration when I find bugs with their software.  I want to tell their devs about the bugs I find.  Software has bugs!  Have a bug tracker! Let people see and understand that software is a continuous improvement process so we get less reviews like this:

firefox feedback in google play store, lamenting the lack of tablet support

I’d love to let Brian know that we are sooooo close to having our tablet support ready, that we have a few outstanding bugs but it’s on-track to ship with Firefox 15 in a mere 7 weeks. We’re a tiny team compared to the Gopplesoft mobile dev teams, give us a chance to prioritize and push each goal to the finish line. With only 20% of our Firefox mobile users on tablets, we had to focus on the 80% small device folks first and then – remember, only 8 weeks later – we got our tablet ducks in a row and ready for our fabulous tablet users.

Alex should get to see a bug filed on the pinch zoom (if there’s not already one) and as one of the admins of the Firefox product, I should get a chance to interact with the folks who leave 1 or 2 star reviews since they are often based on one or two issues that are real but fixable.  I want our rating to be reflective of the work we do as we do it, incrementally improving over time. Of course, our marketplace code is open source so I suppose I should do what Paul Rouget suggested earlier today and make up some prototypes 🙂