App Marketplace Ratings

Woot! Our recently release re-vamp of Firefox for Android is climbing the Top Free chart over on the Google Play Store (as it should, it’s frickin’ awesome). We’ve gone from #96 to #81 in the past 3 days and I have no doubt we will continue to climb as we gain users and get a chance to impress them with the Native UI which is responsive, beautiful, and support Flash. Our rating in the store is also slowly climbing, but that’s going to be a much harder slog because our current rating still reflects the total collected in the entire life of this product being on the store. We can’t remove ratings from our previous Firefox for Android and so even though we’ve had 5,000 5-star reviews in the first 10 days of the re-written version being online, our average rating is a 3.7. The only way to get a fresh start would have been to put up a ‘new’ product and call it something else and I’m sure you can understand that Firefox can’t go by any other name.

I spend a lot of time thinking about the Google Play store and how it could be better because I interact with its administrative backend regularly, uploading new builds of the mobile products release after release. I was just sitting through a presentation by Dees at ReMo Camp 2012 in Berlin and he was sharing with us the strategies behind the Mozilla Marketplace. This will be our open-source contribution to mobile/desktop app distribution and seeing the mockups started me thinking about how we could improve the rating system and not just repeat what Google and Apple do with user feedback.

I’d like to see the following:

User leaves rating, gives stars + writes text feedback
App developer can select reviews to flag as ‘bug report/feedback’ which requires them to write text that will be presented to the user. The developer can write a message either letting the reporter know that a bug is on file now for the issue or provide help with the issues/questions raised by the user.
User gets a notification when the review is flagged and that there is a response ready for them. They can check out the bug report that got filed as a result of their feedback and perhaps they will cc themselves to know when it gets fixed or they might get a chance to try out the suggested solutions from the dev to deal with issues or questions they raised in their original review.
User, now that they have gotten feedback, gets prompted to revise their review.
Repeat 1-4 as needed

This would be beneficial for many reasons:

Users get to be a part of helping improve the product
Users get support from the developer without needing a different forum or login
Users get visibility into software development process, awareness of upcoming features & improvements, and they become participants in open source community
App developers have a channel to communicate with users about upcoming dev plans, feature requests, and bug tracking
App developers get a collected feedback average that is more accurate and representative
App developers have a channel to communicate with users about upcoming dev plans, feature requests, and bug tracking

I’m used to Mozilla’s collaborative environment, the values of open source, and I’m accustomed to getting feedback in our open bug tracker, Bugzilla. There are so many companies whose products I use who do not have public bug trackers and this causes me a lot of frustration when I find bugs with their software. I want to tell their devs about the bugs I find. Software has bugs! Have a bug tracker! Let people see and understand that software is a continuous improvement process so we get less reviews like this:

I’d love to let Brian know that we are sooooo close to having our tablet support ready, that we have a few outstanding bugs but it’s on-track to ship with Firefox 15 in a mere 7 weeks. We’re a tiny team compared to the Gopplesoft mobile dev teams, give us a chance to prioritize and push each goal to the finish line. With only 20% of our Firefox mobile users on tablets, we had to focus on the 80% small device folks first and then – remember, only 8 weeks later – we got our tablet ducks in a row and ready for our fabulous tablet users.

Alex should get to see a bug filed on the pinch zoom (if there’s not already one) and as one of the admins of the Firefox product, I should get a chance to interact with the folks who leave 1 or 2 star reviews since they are often based on one or two issues that are real but fixable. I want our rating to be reflective of the work we do as we do it, incrementally improving over time. Of course, our marketplace code is open source so I suppose I should do what Paul Rouget suggested earlier today and make up some prototypes 🙂

Tomahawk does many things right

Not too long ago I first heard of Tomahawk at one of our Mozilla Monday meetings – they’ve been working with our WebFWD initiative which supports open web projects that are moving the web forward. Tomahawk definitely fits the bill for what the future of the web should be: music just plays. Social, distributed, easy access to tunes through all the open APIs available – I can’t even begin to say how happy I am to be finally able to use this software.

I say ‘finally be able to use it’ because when I first heard of Tomahawk and ran to their site to download it, I stopped in my tracks.

Their original logo was a stereotype cartoon of a North American Indian male, wearing headphones. I’ll let you do your own research if you want to learn why such a depiction isn’t culturally sensitive because that’s not actually the point of this post. The point is that the people behind Tomahawk did constructive criticism a solid. When, as a potential user, I saw this logo and took action by emailing them asking them to consider changing this inappropriate and stereotyping imagery as their masthead and application icon they said other people had contacted them as well, so they were aware and they were WORKING ON A NEW LOGO.

They didn’t get defensive, they didn’t say “we don’t see a problem here”, they didn’t try to justify their choices and protect their precious artwork. They were already working on a solution, continuing forward momentum, adapting as they went, and being civil about it to boot. Who knows how many people contacted them? I might have been the thousandth and yet there was no hint of snark or beleaguered engineer who just wants to write software and not deal with all this soft, people-facing stuff in the response I got from Jason Herskowitz. Thank you for that, Jason and Tomahawk. I really hope this level of maturity catches on in the startup culture.

To give you an idea of how quickly they rotated on this; I emailed on May 14th and the new logo went into place today. That’s slightly more than a month to revamp their site and update their installer and god knows what else (maybe business cards? stationary? do people still use that?). This change isn’t trivial, but neither is the impression that original logo makes about a company. I will proudly go forth now and rally support for this product that I have no doubt is stellar. I’ve been using it for the last 10 minutes since getting the email that the new logo was in place and let me tell you: setup was a breeze. I can also share this experience as a positive one with regards to taking the time to contact a company and point out a culturally insensitive aspect and seeing effective, mature, and expeditious resolution on that issue. This is still rare enough to deserve an entire blog post. I look forward to a future where this sort of thing is a more common dialogue and is always met with positive change and continued forward momentum instead of stop-energy and defensiveness.

Here’s the plug now: Get Tomahawk! It’s open source! It will work with many, if not all, of your current web music accounts! Support the open web and companies that move it forward!

Release-Mgmt: My First Beta from the ‘other’ side

Hello and welcome to my continued documentation off my learning curve in Release Management, something I’ve now been working at for 6 weeks. Last time I was reeling from the new-to-me meeting/email/bugmail firehose. Now I’ve got that more under control, having created many more filters and folders in Thunderbird as well as having a chance to do more work on the automated tracking emails script.

To continue to spread out the knowledge and tasks for release managers across more than just one Alex, last week I ran my first beta (Firefox 12 beta 4) and now I’m going to tell you what that involved because the bugzilla API is down.

Monday

Monday is about getting the queries down as much as possible. This means making sure anything with Tracking?, approval-mozilla{beta,aurora,esr10}? is triaged and nudged further towards its final destination on the trains. We’re also needing to watch out at this point for riskier fixes, things that really need some bake time with actual users need to be landed for a beta4 since beta5 is more for low-risk regression back-outs and security fixes that need to land just in time for beta 6 (what is usually re-built as the appropriately branded final release). At this point in the week there might be about 80-90 bugs we need to get sorted and at the end of the day only about 6 bugs were in the ‘really want this in beta 4’ list.

Tuesday

For this particular beta, we had been asked if it would be possible to give the ‘go’ to build earlier than usual because there was a fairly popular holiday weekend coming up (Easter) and our QA lead for this release was in Canada where Good Friday is a statutory holiday. The QA lead asked if we could go to build earlier so that a Thursday release of the beta would be possible. We confirmed with RelEng that this was do-able and agreed to do our best to get the ‘go’ out at an earlier time. The plan was for everything to be landed by 2:30pm (Pacific) in order to have a changeset ready to fire off to Release Engineering by 5pm.

Going through the ‘burn list’ from Monday (6 bugs) mostly entailed tracking down people to land patches. There has to be a cutoff time for landings since it takes about 4 hours to get all the builds and tests for a push to the hg repo to report back completely. Note that improvements to build times are being worked on, case in point: faster Mac builds (newer hardware and using OS X 10.7) take ~2.5hours off the normal Mac build times.

For one of the bugs we weren’t able to reach the dev and a volunteer committer found that the patch didn’t apply cleanly so that bug had to miss the train. The others got in and I sent the ‘go’ to build Firefox 12 beta 4 at approximately 5:45pm (45 minutes later than desired). All the results weren’t in yet for this changeset but I wanted to get a shot at making the request for earlier builds to QA so I took the risk that the builds/tests would be OK and we’d be already building when we confirmed that. Had the builds/tests *not* turned out we’d have to scrap the chance at moving up the release window and one of our QA leads would have worked on a holiday to meet our ‘normal’ Friday release window. So I took the leap (and this ended up being fine, though I wouldn’t do this again without good reason as it was a stressful call to make and it’s not a good practice to get into).

Right after the ‘go’ email was sent hg.mozilla.org went down and we lost 3 hours of build time. This is not a normal result of giving the ‘go’ to build. It was probably just because Hal was new to doing releases and I was new to running a beta so at least one thing had to come along and shake our confidence.

Wednesday

There’s not much (beta-running) to do on Wednesday except wait for Desktop & Mobile QA to do their thing.

Thursday

Mobile & Desktop QA send their results out – either signing off on the builds/updates or calling out issues. At this point QA signed off so at that point I could request the Release Engineer to push the updates to the beta channel (and upload a new beta apk to the Google Play Store). An hour or so later both QA leads signed off on the updates for the release that’s now live to our beta users. After that, there’s just some product details to change for our websites to include beta 4. We don’t do release notes per-beta which is good to know for when I run a beta 1.

Friday

Normally we’d be doing the push to beta channel on a Friday, so what would have been different was:

getting everything landed to mozilla-beta could have gone until later in the day
the ‘go’ to build email wouldn’t have resulted in immediate builds, they could start on Wednesday at the beginning of the Release Engineer’s day
QA needs two full nights of testing, so we’d get sign offs from QA on Friday morning instead – hence the push to beta channel (and store) on Friday afternoon

Nothing too crazy for my first beta. I think beta 4 is a good one to start on – it’s not the “OMG last call!!1!” beta before a release and it’s not beta 1 where any fallout from our merge of mozilla-aurora -> mozilla-beta shake loose. Having done a beta release now, I have a much more complete mental map of how the 6 week release cycle plays out for Release Management:

Multiply by the above by 6, sprinkle extra bug & meeting communication cycles to weeks 1 and 6, throw in twelve channel meetings, approx 30 more iterations of various queries triage to keep our tracking lists up to date and to know what’s really needing attention vs. what’s taking care of itself.

A ton of email/irc/automated notifications all with the goal of keeping tracked bugs moving forward and you’ve got your 6 week result: A fabulous new Firefox release.

Thanks for reading, more about automatic emails & wiki updates soon.