May 20, 2012 10:11 PM

Yesterday we had BangPypers meeting at ZeOmega office.  There was 10 members came for the meeting.  There was no specific agenda for the meeting, we discussed some general topics related to Python.


I demonstrated the installation of Salt in Windows XP ( http://saltstack.org/ ). Salt is a remote execution and configuration management tool.  For those who missed, here is the screencast I created today for the installation of Salt in Windows: http://www.youtube.com/watch?v=eeJByb-alz8 BTW, community is working on a unified installer for Windows.


If you are interested to learn more about Salt, look at the excellent documentation here:
http://salt.readthedocs.org/en/latest/index.html


You can replace remote execution systems like Fabric & Capistrano with Salt. Also you can replace configuration management systems like Puppet, Chef & CFEngine.


I have tried Salt with RHEL,CentOS,Debian,Ubuntu,Fedora,FreeBSD,Windows 2008 Server R2
and Windows XP.  For example, if you want to install Salt in a CentOS machine, just run
these two commands:


 rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-6.noarch.rpm
 yum install salt*


Similarly for Ubuntu:


add-apt-repository ppa:saltstack/salt
apt-get update
apt-get install salt-master salt-minion


While talking about Salt, I also happened to demonstrate Jenkins server I setup for the
same project ( http://jenkins.saltstack.org/ ).  Here is the screencast I created sometimes
back for the same project: http://www.youtube.com/watch?v=7IRzPFYtyD4
This screencast walk through various code metrics available for Python like:
clonedigger, pep8, pyflakes, pylint & sloccount.

May 20, 2012 07:43 PM

We just released LFS 0.7.4. This is a yet another bugfix release of the 0.7 branch.

May 20, 2012 05:44 PM

We just released LFS 0.6.17. This is a yet another bugfix release of the 0.6 branch.

May 20, 2012 05:44 PM

. in your regular expression

I only vaguely remember my first encounter with a parser generator which must by dated back to the late 1990s. I guess it was Spark by John Aycock, an Early parser. What puzzled me back then was the need to be explicit down to the character level.  Regular expression engines, albeit cryptic, were a revelation because one could specify the structural information one needed and match the rest using a ‘.’ which is the wildcard pattern that matches any character.

I came back to Any in the Trail parser generator lately. I was motivated by writing a reversible C preprocessor. Unlike conventional C preprocessors which are used in the compilation chain of C code, a reversible C preprocessor can used to refactor C code, while retaining the preprocessor directives and the macro calls. This is basically done by storing the #define directive along with the code to be substituted and the substitution. The substituted code and the substitution are exchanged after the refactoring step, such that it looks like no substitution happened at all.

A comprehensive C preprocessor grammar can be found on the following MSDN site. What is most interesting to us are the following two EBNF productions:

# define identifier[( identifier_opt, ... , identifier_opt )] token-string_opt

token-string :

String of tokens 

The String of tokens phrase this is Any+.

Bon appétit

Suppose one defines two macros

#define min(a,b) ((a)<(b)?(a):(b))

#define max(a,b) ((a)<(b)?(b):(a))

Obviously the defining string of the min macro can be recognized using token-string but how can we prevent that token-string eats the max macro as well? Once in motion token-string has a sound appetite and will eat the rest. The solution to this problem in case of regular expressions is to make Any non-greedy. The non-greediness can easily be expressed using the following requirement:

If S | Any is a pattern with S!=Any. If S can match a character, S will be preferred over Any.

In the production rule

R: ... Any* S ...

R: ... (Any* | S) ...

Non greediness in grammars

Grammars are more complicated than regular expressions and we have to take more care about our greediness rules. To illustrate some of the problems we take a look on an example

R: A B | C
A: a Any*
B: b
C: c

Detecting the trailing Any in A is easy though. We solve the follow/first conflict with a trailing Any by embedding A into R. Embedding strategies are the centerpiece of Trail and they shall not be recapitulated here. Just so much: embedding A in R doesn’t destroy any information relevant for parsing. If A has been embedded Any* will be delimited by B to the right and we can safely apply R without the danger of Any consuming a token ‘b’.

Eventually we have to re-apply our embedding strategy: if A is a rule with a trailing Any and A is embedded in B and B has a trailing Any after this embedding  then B will be embedded wherever possible.

A short experiment

Here is a mini-Python grammar is used to detect Python class definitions.

file_input: (NEWLINE | stmt)* ENDMARKER
classdef: 'class' NAME ['(' Any+ ')'] ':' suite
suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
simple_stmt: Any* NEWLINE
stmt: simple_stmt | compound_stmt
compound_stmt: classdef | Any* suite

Unlike a real Python grammar it is fairly easy to build. All rules are taken from the Python 2.7 grammar but only file_input, suite and stmt remained unchanged. In all other cases we have replaced terminal/non-terminal information that isn’t needed by Any.

May 20, 2012 04:01 PM

LuminanceHDR is an excellent free software project for creating high dynamic range photographs from a collection of exposure-bracketed shots of the same scene — first it aligns your photographs using tools from the Hugin project, and then lets you choose between HDR tone-mapping algorithms to create the final image with. Here are some of my photos from the last year processed with LuminanceHDR's implementation of the Mantiuk '06 algorithm, with parameters of contrast=1.0 and saturation=1.2, all available under CC-BY-SA 3.0. (Since the thumbnails below are small, you might prefer to use this Flickr slideshow to see them up close.)

Volkswagen outside Republik; Cambridge, MA


Coolidge Corner Theater; Brookline, MA


Valentine's Day Sunrise; Cambridge, MA


Schloss Charlottenburg; Berlin


Czech Senát; Prague


Skirts and Pants (after Duchamp); Lincoln, MA


Tower (DC); Lincoln, MA


National Museum; Prague


City of Light; Cambridge, MA

May 20, 2012 12:40 PM

May 20, 2012 11:27 AM

In this post we'll explore some effective presentations techniques using Samsung Galaxy S2 accessories. Use the Droid@Screen (See this blog post) program to display the mobile phone screen on my computer during my presentations. This program works by taking a continues series of screenshots of the phone screen. There are some problems with this design, there is noticeable delay during application interaction and the video performance is not great on my Lenovo X120e Ubuntu computer.

You can not demonstrate audio and video capabilities of the mobile phone using such software. Planning to give a talk about using Firefox Mobile with Android 4 screen-reader and existing solutions doesn't work for me.

Samsung Galaxy S2 HDTV Adapter

This adapter allow you to plug in your phone directly into HDMI large displays and digital projectors. The mobile phone AC charger powers the adapter and no additional audio cable is needed. The Samsung HDMI adapters are available for Samsung Galaxy phones and tablet models.

This works really great if you are planning to setup large screen display in a exhibition booth. Connect the phone to large screen display with HDMI cable and play your video demo in a loop.

Samsung Galaxy S2 Power Pack

The Samsung Power Pack comes handy when phone runs out of battery in the middle of the presentation. It does extend the battery life of the phone during the presentation or an extending application testing session.

Few caveats using these accessories. The HDMI adapter will not work when the phone is attached to the power pack .

This power pack has some issue with power sensing, sometimes it starts and stops charging with annoying beeps every 10 seconds. And you need to press locking mechanism tight to get the phone to charge. Hope someone at Samsung fixes this problem.

May 20, 2012 08:30 AM

Today I discovered one of my friends had returned to blogging. Seized with happiness, I went to leave a “welcome back” comment.

Unfortunately, in the two-plus years since I had last left a comment on her blog, WordPress.com had completely redone comment authorization. Even though the text reads

Fill in your details below or click an icon to log in:

(emphasis added) and I filled in my details rather than clicking on a service icon, WordPress.com decided that, because said email address was also associated with my Gravatar or WordPress.com accounts (both, in this case¹), I would have to sign in to WordPress.com in order to leave a comment.

That’s not the end of the world (although way back when, I had carefully crafted my cookies exceptions list to ensure I was remembered on her blog but not anywhere else in the WordPress universe—there’s nothing more frightening than showing up on a site you’ve never visited before and finding that you’re logged-in in the comments field—and generally free from being tracked by WordPress.com in my travels across the web), if that were where it ended. I would have logged in, had my comment posted, logged out, and gone about the rest of my evening, and you’d never be reading this post.

However, what happened is, without any notification whatsoever, WordPress.com replaced the details I had entered (remember, I entered my name, URL and email address instead of clicking on a service icon) with a reference to my WordPress.com account. So instead of “Smokey” from http://www.ardisson.org/ leaving a comment, “sardisson” with no URL left a comment. Even after I visited my never-used WordPress.com profile and entered http://www.ardisson.org/ as my “Web Address” (“Shown publicly when you comment on blogs and in your Gravatar profile.”), my comment still has no URL. I guess because my blog isn’t actually at WordPress.com, I can’t have a Web Address associated with my comments on WordPress.com sites. As for my name, I can also change my “Public Display Name”, but, once again, doing so didn’t alter my comment. (I can also change my WordPress.com username, which might produce the desired effect—though based on the prior two changes it seems unlikely—but I don’t want to jump through the hoops required to do that, and, besides, I like my username just fine.)

On the one hand, I can understand WordPress.com’s desire to force all commenters to use an account from one of their blessed services (even if I don’t agree with the idea), but in that case, why even allow for the appearance of commenting with any name/URL/email? I can also see an argument for forcing anyone who is trying to comment using a known-to-the-WordPress.com-universe email address to log in, so all comments can be associated with the user profile and aggregated (though, in my opinion, that argument is not one that carries much weight).

But if you’re going to force this correlation on visitors/users,

1. Make it clear the association is going to happen, and don’t offer alternative identification UI that leads users to believe they can still comment using the traditional name/URL/email details, and
2. Realize that users—and here by users I mean people, human beings, flesh-and-blood, your mom, your brother, your best friend from college, real people you know and interact with in person on a daily basis, not some abstract construct called “users”—are going to want to identify themselves differently in different contexts,² so you need let them. Not just let them, but facilitate this choice.

After all, even Yahoo! allows you to have separate “identities” associated with the same account and has allowed you to subscribe to different Yahoo! Groups using different identities for so long I’ve forgotten when they introduced that feature. And, er, I believe Gravatar.com supports exactly that sort of thing, different gravatars for different email addresses (except, I guess, if you want to comment on WordPress.com?). Why can’t WordPress.com comments?

Please, just let me comment on my friend’s blog as “Smokey” from http://www.ardisson.org/ using the email address I customarily use on the internet, and let me choose to comment elsewhere on WordPress.com blogs as “Smokey Ardisson” or “sardisson” or whatever facet of my identity is most appropriate for the context in which I am commenting.

¹ Even if I hadn’t used the same email address on both services, once Auttomatic acquired Gravatar and linked it with WordPress.com, practically speaking for everyone the two accounts are one and the same. ↩

² :cough: Google Buzz :cough: Google Plus :cough: ↩

by Smokey at May 20, 2012 07:29 AM

Whenever I search in Aurora on my phone, I'm taken to a stripped-down version of the page with the header "this page adapted for your browser".

How do I fix this? I'd rather fix it with a Google preference, but barring that I expect that Firefox has a way for me to regain control of my online experience?

May 19, 2012 11:33 PM

Like many FOSS fans, I always wanted to be an active part of the movement. My last big project was for the Amiga in the past millennium though. Nowadays I’m happy that after years of small-scale dabbling on various projects I’ve found my haven. I’d like to share my way to my recent gain of push privileges on the Python project and hope to inspire some of you to do the same.

The Problem

It isn’t easy to start contributing to FOSS. Some projects seem even not to be particularly interested in new contributors – because everyone wants to contribute for fame and glory. Others could use help but aren’t eager to get new patches because of ample amounts of code rotting in the bug tracker. But there are plenty of projects that want you to help.

And yes, even CPython — i.e. the most widespread Python implementation as of 2012, I’ll call it just “Python” henceforth — needs your help! That’s the reason it’s active in its outreach to new contributors. Please note I said “contributors”, you don’t have to be a programmer – or do programmer’s work – to help.

Python core development has always been a welcoming place. But Jesse tried to push things even farther. And he succeeded at that – it’s easily one of the best places to hang out with people way smarter than you (or at least me) without being patronized.

But it’s hard to get started with new code and often even harder to get started with a new community. So what’s the most important personal attribute to get into a high profile open source project like Python, Django or Twisted? Hint: it’s like with everything important in life you can’t buy.

The Solution

Perseverance.

People sense that but don’t believe it. It’s just like kids don’t believe they’ll ever become adults and their parents were where they are now at some point in prehistoric times. And just like that, the now famous CPython developers once started submitting patches to a tracker and prayed for someone to notice.

I’m in no way famous but I think that’s exactly what it makes easier for you to relate to me.

Please compare the dates between my first patch (2011-08-08) to the bug tracker and the day I got my push rights (2012-05-14): nine months. What’s even worse, my first real code patch rotted for four months in the tracker. I’ll be honest with you: I walked away from Python core development at this point.

“If they don’t want my help, I’ll look elsewhere.”

Fortunately, Antoine gave me my “christmas presents” and committed it at last out of nowhere. That made me come back and I’m really happy about that. But was it really necessary for me to leave my ambitions? Not really. I’ll try to draw a picture of the real situation.

The problem is: the delay didn’t indicate that Python isn’t interested in my help. It proved it needs my help very much.

Although there are lots of committers, only few of them are active (try running hg churn -c -d '2012 to 2013' if you don’t believe me). Committing code to a project like Python is different than dabbling on some small project. We have a high responsibility to not break stuff livelihoods depend on. Someone who commits your code has to fully understand what it’s doing and takes responsibility for your patch (although breaking stuff isn’t really a crime as long as it breaks loudly, that’s what we have buildbots for).

So, logically few active committers means few checkins means long delays on patches.

So in the beginning, you really need some patience. It also helps to hang out and socialize on Freenode’s #python-dev or the Python mailing lists (although I’m rather shy on posting on mailing lists myself; it doesn’t feel good to ask basic stuff that gets archived forever). Your also gain good karma but reviewing code of others and generally being active on the bug tracker. Once some of your patches have been committed, other developers will be much more open to look at your code. We’re all human in the end and you would also rather help out a friend, no?

And that would have been the better way for instead of retiring. While I was on IRC, I wasn’t really active on the tracker building up reputation. After my first patches were committed and I started triaging and reviewing, people even started asking me whether I need to have something reviewed.

When you arrive at this point, you’re over the hump. Contributing becomes much less of a insular patience game and much more of a social experience. You’re part of a community hacking on something cool – and while there’ll always be friction where people gather and you’ll still have to wait for reviews – it’s a lot of fun.

And one day – out of the blue – you’ll get extended privileges offered. At this point it’s important to keep your cool; too many people started doing way too much, burning out and stopping doing anything at all.

What Python Needs Most

We need triagers that weed through tickets – new and old – and help to close them. Be it by reviewing code, checking spelling, grammar and wording, trying to reproduce the problems – we’re especially in need of capable Windows users – or just pinging stale tickets and adding suitable developers to the watch lists. There’s a lot to do and the current team is glad to be able keep up with new stuff, but there are 1,500 and growing tickets with patches open – and I’d like very much to change that.

Let me stress at this point that it’s unlikely that you get commit rights if you just add own patches and wait for them to be committed. It’s expected from you to take part at the whole development process.

So my foremost priority for the next months is doing exactly that: cut back on own coding and weed through tickets and make people happy by getting simple stuff merged or at least noticed. I know from personal experience that ignored patches are worse than rejected ones.

But I could use the help of people like you, who speak Python or English and are willing to learn Python’s development process! Get an account on the tracker, familiarize yourself with our triaging process and start triaging! Meanwhile, feel free to ask your questions on Freenode’s #python-dev or our core-mentorship mailing list.

Your efforts will be noticed. And nothing’s as fun as breaking all buildbots with one commit. :)

May 19, 2012 06:35 PM

I decided I had to write my own Document Object Model implementation because the models available to me just won’t meet my requirements.  That involves writing a lot of code, though, and ensuring that Element nodes, Text nodes, etc. all implement Node properties like firstChild, childNodes, etc., and methods like appendChild() correctly.

To lend a hand, I wrote a little InterfaceChecker library for ensuring that tests on a base class are run on derived classes.  (I know, JavaScript uses prototypes instead of proper classes; I’m using the concept’s meaning here.)  In principle, I need to only do three things:

• Write tests for the Node interface’s methods and properties
• Designate Element as inheriting from Node, and
• Provide some functions to build “typical” instances of Element,

My InterfaceChecker will run the Node tests against the “typical” instances of Element, for example.  It’s not DOM-specific, either:  I can do the same to arbitrary JavaScript constructors, like a theoretical Shape and Circle pairing.

This code is for debugging purposes only; in my opinion, it’s a little extreme to include these tests in a production environment.

Please, let me know what you think!

by ajvincent at May 19, 2012 05:07 PM

It is true, from time to time I read things on the internet. What has been bothering me lately is the feeling that all of these ideas I'm consuming and all of my responses to them are just lost into a sea churning together everything into the unidentifiable paste that comes out of my head.


So, I've decided to start logging this. After all, the word "blog" is a shortening of "weblog", a web log, a log of the things you've read on the web. I want this for my own daily readings, both to keep a record for myself (and anyone with an interest) and to note my thoughts, comments, responses, and questions about the things I come across.


Ironfroggy Reads Things On The Internet is this new thing. It is a blog on Tumblr, which I find a useful tool for smaller posts and their bookmarklet is perfect for my needs here. I'll be directing the posts to twitter to, as an experiment. Let me know if that turns out annoying.


Follow it, if you care to. Mostly its for me, but I believe in transparency by default.

May 19, 2012 05:02 PM

Pretty often I point bug reporters at random git branches and sometimes they'll happily compile kernels from sources, but have no idea about git. Unfortunately there doesn't seem to be a quick how-to tailored to the needs of bug reporters, so let's fix that.

$ git clone --depth 1 --no-checkout --no-single-branch <git-repo-path>

This clones the git repository, but avoids downloading the entire history. It also avoids to check out the default branch because usually you need a specific branch for testing.

All the following commands only work if the working directory is within the newly cloned git repository.

$ git checkout origin/<branch-name>

will check out the git branch <branch-name> (from the remote git repository) into the local git repository so that you can use it. git will complain about 'detached HEAD' but that's only really important when committing new changes, so just ignore that. If you already have an older clone of the git same remote repository, you first need to update the local git database (in the .git directory in your local repository) with

$ git fetch origin

before checking out the branch again. Even if you need to test the same, but updated branch, you need to do both steps - git doesn't update anything automatically by default.

If you have patches or other local changes applied, git might complain that there's a conflict. You can simply remove all local changes with

$ git reset --hard

and then re-do the checkout command. Much more rarely some build artifacts could get in the way (or you just need a quick way to start a clean build),

$ git clean -dfx

will remove any files not in the currently checked-out git commit.

May 19, 2012 04:51 PM

I needed to add a boolean field to a model. For many web apps, this typically involves:

1. modifying the model layer, so that the field becomes available as an attribute on retrieved objects, and can be queried against etc.
2. creating a database migration script that can be run immediately on the development box, and later for staging and production.
3. running the migration against the development DB.
4. updating any admin screens for editing the field.
5. checking the changes and scripts into source control.
6. deploying - including pushing source code and running migration scripts etc.

Using Django, from a cold start (no editor/IDE open), this just took me 1 minute 45 seconds of work for steps 1 - 4, and an additional 45 seconds waiting for step 5, total 2 minutes 30 seconds, and I wasn't rushing.

Step 1 is a one line code addition. Pretty much everything else can and should be generated automatically.

Step 2 is taken care of by a one line command using South, as is step 3 and the database part of step 6 (which is run de-rigueur from my deployment scripts).

Step 4 is taken care of by Django's admin, which introspects the model and generates the right form for you.

This is one of the reasons I love Django. It's not so much the time it saves, although that is pretty awesome, it's the tedium it saves.

This is also one of the reasons I'm not very tempted by schema-less or schema-light databases, because with Django a nice strict schema brings so little administrative overhead. I was going to have to add something about the change to the model anyway, even if it was only documentation, and having done that in one place, the other additional changes required by a relational DB with strong schema placed virtually no burden on me.

(Of course, things could be more complex on bigger apps, especially if the table is large or sharded. But then again, there's no reason why rolling out your DB change shouldn't be just as automated - it's only the 'waiting' stage that has to take longer for a simple change like adding a column. If the coding/work part is taking much longer than the above example, your tools probably need fixing or replacing.)

May 19, 2012 02:12 PM

Twisted is excited to be supporting 4 full-time summer interns from around the world through 2 internship programs this summer.

Google Summer of Code internships

Expanded Endpoints Support, by Ashwini Oruganti (IRC nick ashfall)

Ashwini joins us from the Manipal Institute of Technology in Manipal, India. She has already worked on and closed out a number of Twisted tickets and has previously contributed to Evolution and Sugar Labs.

Her project:

Recently, two new APIs, IStreamServerEndpoint and IStreamClientEndpoint were added to Twisted, for specifying what address the servers should listen for connections and what address a client should connect to, respectively. But not all of the addresses that Twisted supports have this endpoint support added to them; presently endpoint support has been implemented for TCP, SSL and UNIX domain sockets. My project deals with adding more endpoint implementation to Twisted, some involving wrappers around the existing APIs (e.g. serial ports, standard I/O), others involving making fresh APIs where setting up connections was difficult before the addition of the endpoints (e.g. SOCKS and HTTPS proxies).

Python 3 preparation, by Vladimir Perić (IRC nick vperic)

Vladimir joins us from Czech Technical University in Prague. Last year he was a Google Summer of Code student with SymPy.

His project:

Python 3 is the future of Python. If Twisted is to see continued usage in the future, it will have to be ported, and rather sooner than later. As Twisted is a large and complicated code-base, this process needs to be done with care, ensuring that any code written remains compatible with the currently supported versions. The test-driven development methodology Twisted uses will ensure no regressions happen and will ease the maintenance of the code-base.

Automatic Coding Standard Enforcement, by Raphael Shu (IRC nick zomux)

Raphael joins us from Tsukuba University in Japan, where he uses Python daily in his NLP research.

His project:

Twisted applies certain naming and style standards to all contributed code. Currently, a human reviewer needs to check all of these things. The purpose of this project is to develop a tool which can automatically make these simple, mechanical checks, freeing up human reviewer time to focus on more important aspects of proposed changes. Finally, it will speed up the review process.

Software Freedom Conservancy / GNOME Outreach Program internship

We are also excited to be working with the Software Freedom Conservancy and the GNOME Outreach Program for Women this summer. You can read more about the initiative and our work to encourage diverse participation in open source communities here. Through this initiative we have a 4th paid, full-time internship this summer:

Improving Twisted Mail and Twisted Core, by Fei Tan (IRC nick argonemyth)

Fei joins us from Grand Bay, Mauritius, where she works as a freelance web developer.

She will improve Twisted Mail on a number of fronts, including improving API documentation, adding more examples, adding more HOWTOs, and improved test coverage.

Please join me in welcoming Ashwini, Vladimir, Raphael, and Fei, whose internships start next week. Expect a torrent of code reviews and some record-breaking high scores list stats this summer!

Thank you Google for giving us this paid mentorship opportunity, and thank you to the Python Software Foundation for supporting us as our Google Summer of Code umbrella organization.

May 19, 2012 11:08 AM

I recently forked the defunct eyefiserver project. The new project is called eyefiserver2. It is a server for eye-fi cards that runs on Linux using Python, however, it should be possible to run it on any OS, I just haven't tested it on anything other than Linux.

May 19, 2012 07:17 AM

For those who labor long and hard to craft good and just standards, as well as those who have suffered from their absence. On the one hand, the fight against the tyranny of structurelessness. On the other, the fallacy of one size fits all  (Lampland & Starr, 2009).

This book dedication found in Standards and Their Stories captures the inherent paradox of badge system design. By seeking to standardize the process we risk the introduction of systemic rigidity. And yet by developing badges without a plan we risk the possibility of ideological entropy. In my writing about this topic I’m attempting to walk the middle path: somewhere in between fanatical dictums and a mad free-for-all. I wish I could say that it was easier than this, but then I’d be lying.

The status quo
Even while we’re in the midst of talking about a potentially reconstructive idea like Mozilla Open Badges, I still rather rotely refer to my own typically conventional educational route with “my undergrad degree this” or “my grad degree that.” Perhaps this is to be expected. It certainly hearkens to one of the issues that the open badges in the wild will have to confront: the seeming intractability of the status quo. In the Open Badges world this desire for stability echoes within the repeated request for a standard method of validation; it’s mated to a deep concern about badge quality. In unfamiliar situations such as these we tend to rely on current cultural understandings and touchstones. In this case, degrees and certificates, accreditation systems and educational rankings.

The status quo of our formal academic system has transmogrified into a sort of binary approval system. You pass or you fail. You go to a respected school or you go to a second-tier school. You graduate or you don’t. It all seems pretty inexorable. We gravitate toward that which is customary. The familiar often appears to be less threatening than the entirely unknown. Indeed, there is a robust academic research field that studies this tendency, especially with regards to our proclivities toward risk and reward: behavioral economics. (For a deep and delightful dive on this read Dan Ariely’s Predictably Irrational.)

Resonance
I’m hoping that some day people will refer to not only their formal schooling but their non-traditional learned experiences as well (hopefully badged in the open way) without speaking of one of them as second-rate or less than the other. That noted, I’ll return to my rather classical undergraduate education to make a point. I double-majored in graphic design and writing. The classes I took in design inform a significant amount of the way that I think. This is not to say that every design class I took made sense or built on every preceding design class so that one day I had taken enough of them to—ta-dah!—be called a designer. On the contrary, I gleaned information from a variety of sources. My deep learning occurred in many different venues, a bit of it very much outside the realm of what typically would be called design. Nevertheless, some aspects of design that I learned in those college classes continue to reverberate within me.

One of the most resonant aspects of those years pertains to users and audiences and owners and consumers and interested parties and even uninterested parties. The idea of multiple audiences pulses within me at the root. Akin to that concept, another: juxtaposition. What is there versus what is not there; what has been asked versus what has not been asked; the solid versus the void. Good designers are problem solvers, not stylists or skinners. They interrogate situations and ask why? They poke around in seemingly unrelated categories. They consider the issue of temporality and end users while acknowledging that a problem owner need resolutions. They know that solutions can have many audiences and that things that seem straightforward can be damn complex. (Massimo Vignelli has spoken eloquently on this subject in “Massimo Vignelli on Rational Design.” Actually, read all the interviews on Steven Heller’s Design Dialogues site.)

Hard questions
Why do I mention all of this? Because as you begin the process of badge system design, you, too, will be delving into these areas. You, too, will be learning to be a designer. You’ll be gathering information from many sources—no doubt a few of them entirely unexpected. And most likely you’ll find yourself asking deep and sometimes existential questions. I encourage you to remain open to the idea that periodically, like the question, the answer will prove to be both complex and difficult and very much not binary. Sometimes you will have to try something to know if it works because there will be no answer until you do. Accept this. Your badge system will benefit from this sideways approach. That is, believe it or not, the middle path.

- – -

Much more soon.

references
Ariely, D. (2008). Predictably Irrational. New York, NY: Harper Collins.
Lampland, M. & Starr, S. L. (2009). Standards and their stories. (p. dedication). Ithaca, NY: Cornell University Press.
Vignelli, M. (1998). Massimo Vignelli on Rational Design. In Heller, S. (Ed.), Design dialogues (pp. 3-8). New York, NY: Allworth Press.

by carlacasilli at May 19, 2012 01:58 AM

I’ve had some requests to port Cleary over to Native Fennec (aka the hot new Beta for Android), and thought it would be a good chance to blog about addons in the native Fennec world. Cleary is a pretty simple addon (based on mfinkle’s BootstrapJones tutorials) that gave Fennec some more advanced clear history options, like the ability to only clear certain things:

1. History (cache,history,downloads,formdata)
2. Passwords (cookies,passwords,sessions,syncAccount)
3. Site Preferences (offlineApps,geolocation,siteSettings)

and only clear them for certain time frames (One hour, Two hours, Four hours, a Day, and Forever). Porting it gives a chance to show off some new things we can do. Turned out, porting it was really easy though. In fact, a lot of the code it was using in XUL Fennec isn’t even necessary anymore thanks to some additions to the platform! Quick overview:

Step One: Updating compatibility

This steps pretty easy. We just need to mark Cleary as compatible on our install.rdf

<em:targetApplication>
  <Description>
    <em:id>{aa3c5121-dab2-40e2-81ca-7ea25febc110}</em:id>
    <em:minVersion>14.0b1</em:minVersion>
    <em:maxVersion>15.0</em:maxVersion>
  </Description>
</em:targetApplication>

Updating our UI

Cleary (luckily), is a bootstrapped extension and doesn’t include any overlays. While you could potentially use an overlay in Native Fennec, I’d highly discourage addon authors from doing it.

So to update our UI here basically means altering some scripts so that they only run on XUL Fennec. I decided an easy way to do that here was just to check the app id:

if (Services.appInfo.ID == "{a23983c0-fd0e-11dc-95ff-0800200c9a66}") {
  // setup xul fennec
} else {
  // do nothing!
}

Inside here the XUL Fennec code creates some buttons and rows for the Preferences pane. Overlaying things into prefs isn’t possible in Native yet, so we can basically do nothing! In some cases you might opt to use our NativeWindow interface to add UI elements. Instead, I’m opting to include some inline options for the addon.

settings.xul also happened to already be in Cleary. I just used some trickery to display it in the normal settings list. It also included some things we didn’t want, like a <settings> element, which I’m pretty sure Native Fennec will not like, so I removed that and just created a new options.xul in the root of the addon.

We also need to re hook up our click listeners to the clear buttons in the dialog. This requires a fix for Fennec that hopefully we can push into the beta. In bootstrap.js we need to add a listener for “addon-options-displayed” notifications:

Services.obs.addObserver(observer, "addon-options-displayed", false)

and a little observer to see them trigger:

var observer = {
  observe: function(aSubject, aTopic, aData) {
    if (aTopic == "addon-options-displayed" && aData == "cleary@digdug.org") {
      var doc = aSubject;
      var control = doc.getElementsByClassName("clearButton");
      for (var i = 0; i < control.length; i++) {
        var c = doc.defaultView.gChromeWin.cleary;
        control[i].addEventListener("command", c.doClear.bind(c), false);
      }
    }
  }
};

The old settings.xul was also localized using some JS trickery, which won’t work now. BUT!!! bootstrapped addons now support a chrome.manifest file, so we can add it and save ourselves some work. At the same time, I can remove a bunch of junk I had to register a resource URL and load style sheets. Yay for simplifying! We just add a chrome.manifest file with:

content cleary content/
locale cleary en-US locales/
style chrome://browser/content/browser.xul chrome://cleary/content/dialog.css

Functionality

Most of the functionality we previously had will just work in Native Fennec. Passwords, Form history, Cache, Cookies, Downloads, etc. All of its the same (and the same as desktop).

Except! browsing history is no longer handled in Gecko at all. Its all done in Java. Eventually Fennec should expose an interface for this, ideally that one that mirrors nsIBrowserHistory. I filed bug 731888 awhile ago for a slightly less hacky fix as well, but need to put up a correct patch there (volunteers welcome!).

For now, I had to hack into the database. This is NOT recommended. It will probably break eventually, and it will definitely make kittens cry but you can open the history database using the mozStorage service and tinker around in it. That may crash some phones where Android’s sqlite assuredly won’t match Gecko’s, so be warned! For history I added:

let file = FileUtils.getFile("ProfD", ["browser.db"]);
let dbConn = Services.storage.openDatabase(file);
if (this.range) {
  var statement = dbConn.createStatement("DELETE FROM history WHERE :from_date <= date AND date <= :to_date "
  statement.params.from_date = this.range[0];
  statement.params.to_date = this.range[1];
  statement.executeAsync();
} else {
  dbConn.executeSimpleSQL("DELETE FROM history");
}

And we’re done! There’s a build with this on Dropbox here, but it will not work until bug 756689 is fixed an checked in (and be warned its not on AMO because I’m still testing! Dataloss definitely possible!) There’s some bugs here I need to look into (clicks in the options dialog seem to fire multiple times and there’s some oddity with menulists), and I haven’t tested on XUL Fennec yet. For things that delete your data, there’s always lots and lots and LOTS of testing to do. But for the most part the port was pretty easy.

I’m pumped about Native Fennec. Its a great browser, and having extensions work in it is just icing on the cake. I’m excited to see what some good add-on developers can come up with to do with it in order to fix and enhance the mobile web.

by digdug2k at May 19, 2012 01:43 AM

A few months ago, I "took over" maintenance of elasticutils. We use it in SUMO as the API for building search queries with elasticsearch.

One of the first things I did was spend some time figuring out whether we should keep working on elasticutils at all. django-haystack also provides a django-ish API for working with elasticsearch. Why have two libraries that at a high level do the same thing?

The thing is that they're not exactly the same. django-haystack is really great and supports a variety of backends for search, elasticsearch being one of them. Right now, it only has support for elasticsearch in 2.0 which is in either an alpha or beta state now (their web-site could use some updates). However, because it supports a bunch of backends, it only supports functionality that works across all of them.

elasticutils, on the other hand, is elasticsearch-specific. As elasticsearch adds functionality, we can, too. That's the compelling reason to keep working on this library. However, django-haystack has some awesome ideas that we'd like to implement in elasticutils, too. This will fix some sharp edges in elasticutils, but also make it much easier for projects to switch from one to the other.

Currently, elasticutils only handles the query side of things. django-haystack handles that, but also has an API for defining mappings, indexing, and all the other things you need with a search system.

Thus, Rob Hudson and I are going to embrace and extend elasticutils to:

1. fix the current situation where it seems every elasticutils user is actually using their own branch with additional functionality in it (ew!)
2. implement the rest of the things you need with a search system
3. document the things we've learned while working with elasticutils because at a minimum, it seems most of the Mozilla projects that use elasticutils bumped into, spent time on, and solved the same problems---that's a huge waste of time and a failure on my part

One of the things users of a library need is for the library to be a mature project with releases, tagged version, documentation, tests, stability, reliability, reproduceability, communication, community and all that. Thus, I'm also going to spend some time to turn this into a real project. Towards that end, I created #elasticutils on irc.mozilla.org where we'll talk dirty elasticutils stuff. If we end up with more people pitching in, we'll create a mailing list. But for now, IRC will do.

My next step is to spend a little time cleaning up what's in the master branch, then tag and release a baseline version.

After that, I'm going to spend time identifying, thinking about and merging in the divergent functionality in the various branches while Rob works on continuing his imperative mapping work.

I think in a couple of months, we'll be in a better place and that'll make it easier for Mozilla projects and anyone else who wants to use elasticutils to use and contribute to it.

If you're a user of elasticutils, please come hang out with us! Let us know how we can better help you.

[Comments]

May 19, 2012 12:01 AM

(Note: the following has been cross-posted to Mozilla UX)

Two Firefox features getting a redesign in Firefox 13 (currently in beta) are the Home Tab and New Tab. Home Tab can be viewed by clicking house icon in Firefox or by typing “about:home” into your URL bar. New Tab appears when you click the “+” at the end of your tab strip.

Firefox 13 New Tab Page

Firefox 13 Home Tab Page (launch targets emphasized)

Firefox’s Home Tab and New Tab have, until now, had fairly basic pages. In Firefox 12, Home Tab had a large search bar, a “snippet” which Mozilla uses to display messages to users, and little else. The main reason the search bar is on Home Tab is because many users click the Home button to initiate a search, either unaware of the toolbar search box or preferring not to use it. The snippet allows Mozilla to give a message to users, such as last October when it asked users in the United States to contact their representatives when the anti-internet-freedom bill SOPA was being heard in the House of Representatives. Such messages can be important while not being urgent enough to disrupt users with a notification.

New Tab, for most of Firefox’s history, has been completely blank. This was done deliberately to offer users a clean, fresh “sheet” to begin a new browsing task. However, a blank tab may not be distracting, but it’s also not useful.

Surely, we thought, we can present a more helpful design than a blank page! Using Mozilla Test Pilot, we began to research how Firefox users use New Tabs. What we learned is that each day, the average Firefox user creates 11 New Tabs, loads 7 pages from a New Tab, and visits two unique domains from a New Tab. The average New Tab loads two pages before the user closes or leaves it.

What this tells us is that users create many New Tabs, but they’re very likely from those to return to a limited number of their most-visited websites. So, we began to experiment with giving users quick access on New Tab to the websites they visit most frequently.

What you’ll see on the New Tab page of Firefox 13 are your most-visited sites displayed with large thumbnails, reducing the time it takes to type or navigate to these pages. This data comes directly from your browsing history: it’s the same information that helps Firefox’s Awesome Bar give suggestions when you type. Or, if you want to go somewhere new, the URL bar is still targeted when you type on a New Tab page. If you want to hide your top sites – permanently or temporarily – a grid icon in the top right wipes the new tab screen to blank.

Mozilla Home is getting a redesign, too! While still keeping the prominent search bar and snippet, the graphic style is softer, the text is more readable, and launch targets at the bottom allow you to quickly access areas such as Bookmarks, Applications, and previous Firefox sessions.

Both Home and New Tab are being improved as part of our longterm vision of making Firefox more powerful, engaging, and beautiful. Over the next few releases, more design improvements will be made towards this goal. For now, please try out Firefox’s new Home and New Tab pages in Firefox 13 Beta and tell us that you think!

by Jennifer Morrow at May 18, 2012 11:43 PM

Two Firefox features getting a redesign in Firefox 13 (currently in beta) are the Home Tab and New Tab. Home Tab can be viewed by clicking house icon in Firefox or by typing “about:home” into your URL bar. New Tab appears when you click the “+” at the end of your tab strip.

Firefox 13 New Tab Page

Firefox 13 Home Tab Page (launch targets emphasized)

Firefox’s Home Tab and New Tab have, until now, had fairly basic pages. In Firefox 12, Home Tab had a large search bar, a “snippet” which Mozilla uses to display messages to users, and little else. The main reason the search bar is on Home Tab is because many users click the Home button to initiate a search, either unaware of the toolbar search box or preferring not to use it. The snippet allows Mozilla to give a message to users, such as last October when it asked users in the United States to contact their representatives when the anti-internet-freedom bill SOPA was being heard in the House of Representatives. Such messages can be important while not being urgent enough to disrupt users with a notification.

New Tab, for most of Firefox’s history, has been completely blank. This was done deliberately to offer users a clean, fresh “sheet” to begin a new browsing task. However, a blank tab may not be distracting, but it’s also not useful.

Surely, we thought, we can present a more helpful design than a blank page! Using Mozilla Test Pilot, we began to research how Firefox users use New Tabs. What we learned is that each day, the average Firefox user creates 11 New Tabs, loads 7 pages from a New Tab, and visits two unique domains from a New Tab. The average New Tab loads two pages before the user closes or leaves it.

What this tells us is that users create many New Tabs, but they’re very likely from those to return to a limited number of their most-visited websites. So, we began to experiment with giving users quick access on New Tab to the websites they visit most frequently.

What you’ll see on the New Tab page of Firefox 13 are your most-visited sites displayed with large thumbnails, reducing the time it takes to type or navigate to these pages. This data comes directly from your browsing history: it’s the same information that helps Firefox’s Awesome Bar give suggestions when you type. Or, if you want to go somewhere new, the URL bar is still targeted when you type on a New Tab page. If you want to hide your top sites – permanently or temporarily – a grid icon in the top right wipes the new tab screen to blank.

Mozilla Home is getting a redesign, too! While still keeping the prominent search bar and snippet, the graphic style is softer, the text is more readable, and launch targets at the bottom allow you to quickly access areas such as Bookmarks, Applications, and previous Firefox sessions.

Both Home and New Tab are being improved as part of our longterm vision of making Firefox more powerful, engaging, and beautiful. Over the next few releases, more design improvements will be made towards this goal. For now, please try out Firefox’s new Home and New Tab pages in Firefox 13 Beta and tell us that you think!

by Jennifer Morrow at May 18, 2012 11:43 PM

The other day, I thought it would be fun to create a little program that could generate QR codes and show them onscreen with wxPython. Of course, I wanted to do it all with Python, so after a little looking, I came across 3 candidates:

• python-qrcode on github
• pyqrcode on sourceforge and
• pyqrnative on Google code

I tried python-qrcode and pyqrnative since they worked on Windows as well as Mac and Linux. They also didn’t require anything except the Python Imaging Library. The pyqrcode project requires several other prerequisites and didn’t work on Windows, so I didn’t want to mess with it. I ended up taking some old code based on my Photo Viewer application and modified it slightly to make this a QR Code viewer. If I’ve piqued your interest, then read on!

Getting Started

As I noted at the beginning, you will need the Python Imaging Library. We’ll be using wxPython for the GUI part, so you’ll need that as well. And you’ll want to download python-qrcode and pyqrnative. The main difference I have found is that python-qrcode is much faster at generating the images and it generates the type you’ve probably seen the most of. For some reason, pyqrnative take a lot longer to run and it creates a much denser looking QR code. There may be options for both of these projects that allow you to generate different kinds of codes, but the documentation for either project is abysmal. I ended up using the source and Wingware’s IDE to traverse the code more than anything else.

Generating QR Codes

Anyway, once you have all the prerequisites, you can run the following code and see what Python can do:

import os
import wx
 
try:
    import qrcode
except ImportError:
    qrcode = None
 
try:
    import PyQRNative
except ImportError:
    PyQRNative = None
 
########################################################################
class QRPanel(wx.Panel):
    """"""
 
    #----------------------------------------------------------------------
    def __init__(self, parent):
        """Constructor"""
        wx.Panel.__init__(self, parent=parent)
        self.photo_max_size = 240
        sp = wx.StandardPaths.Get()
        self.defaultLocation = sp.GetDocumentsDir()
 
        img = wx.EmptyImage(240,240)
        self.imageCtrl = wx.StaticBitmap(self, wx.ID_ANY,
                                         wx.BitmapFromImage(img))
 
        qrDataLbl = wx.StaticText(self, label="Text to turn into QR Code:")
        self.qrDataTxt = wx.TextCtrl(self, value="http://www.mousevspython.com", size=(200,-1))
        instructions = "Name QR image file"
        instructLbl = wx.StaticText(self, label=instructions)
        self.qrPhotoTxt = wx.TextCtrl(self, size=(200,-1))
        browseBtn = wx.Button(self, label='Change Save Location')
        browseBtn.Bind(wx.EVT_BUTTON, self.onBrowse)
        defLbl = "Default save location: " + self.defaultLocation
        self.defaultLocationLbl = wx.StaticText(self, label=defLbl)
 
        qrcodeBtn = wx.Button(self, label="Create QR with qrcode")
        qrcodeBtn.Bind(wx.EVT_BUTTON, self.onUseQrcode)
        pyQRNativeBtn = wx.Button(self, label="Create QR with PyQRNative")
        pyQRNativeBtn.Bind(wx.EVT_BUTTON, self.onUsePyQR)
 
        # Create sizer
        self.mainSizer = wx.BoxSizer(wx.VERTICAL)
        qrDataSizer = wx.BoxSizer(wx.HORIZONTAL)
        locationSizer = wx.BoxSizer(wx.HORIZONTAL)
        qrBtnSizer = wx.BoxSizer(wx.VERTICAL)
 
        qrDataSizer.Add(qrDataLbl, 0, wx.ALL, 5)
        qrDataSizer.Add(self.qrDataTxt, 1, wx.ALL|wx.EXPAND, 5)
        self.mainSizer.Add(wx.StaticLine(self, wx.ID_ANY),
                           0, wx.ALL|wx.EXPAND, 5)
        self.mainSizer.Add(qrDataSizer, 0, wx.EXPAND)
        self.mainSizer.Add(self.imageCtrl, 0, wx.ALL, 5)
        locationSizer.Add(instructLbl, 0, wx.ALL, 5)
        locationSizer.Add(self.qrPhotoTxt, 0, wx.ALL, 5)
        locationSizer.Add(browseBtn, 0, wx.ALL, 5)
        self.mainSizer.Add(locationSizer, 0, wx.ALL, 5)
        self.mainSizer.Add(self.defaultLocationLbl, 0, wx.ALL, 5)
 
        qrBtnSizer.Add(qrcodeBtn, 0, wx.ALL, 5)
        qrBtnSizer.Add(pyQRNativeBtn, 0, wx.ALL, 5)
        self.mainSizer.Add(qrBtnSizer, 0, wx.ALL|wx.CENTER, 10)
 
        self.SetSizer(self.mainSizer)
        self.Layout()
 
    #----------------------------------------------------------------------
    def onBrowse(self, event):
        """"""
        dlg = wx.DirDialog(self, "Choose a directory:",
                           style=wx.DD_DEFAULT_STYLE)
        if dlg.ShowModal() == wx.ID_OK:
            path = dlg.GetPath()
            self.defaultLocation = path
            self.defaultLocationLbl.SetLabel("Save location: %s" % path)
        dlg.Destroy()
 
    #----------------------------------------------------------------------
    def onUseQrcode(self, event):
        """

https://github.com/lincolnloop/python-qrcode

        """
        qr = qrcode.QRCode(version=1, box_size=10, border=4)
        qr.add_data(self.qrDataTxt.GetValue())
        qr.make(fit=True)
        x = qr.make_image()
 
        qr_file = os.path.join(self.defaultLocation, self.qrPhotoTxt.GetValue() + ".jpg")
        img_file = open(qr_file, 'wb')
        x.save(img_file, 'JPEG')
        img_file.close()
        self.showQRCode(qr_file)
 
    #----------------------------------------------------------------------
    def onUsePyQR(self, event):
        """

http://code.google.com/p/pyqrnative/

        """
        qr = PyQRNative.QRCode(20, PyQRNative.QRErrorCorrectLevel.L)
        qr.addData(self.qrDataTxt.GetValue())
        qr.make()
        im = qr.makeImage()
 
        qr_file = os.path.join(self.defaultLocation, self.qrPhotoTxt.GetValue() + ".jpg")
        img_file = open(qr_file, 'wb')
        im.save(img_file, 'JPEG')
        img_file.close()
        self.showQRCode(qr_file)
 
    #----------------------------------------------------------------------
    def showQRCode(self, filepath):
        """"""
        img = wx.Image(filepath, wx.BITMAP_TYPE_ANY)
        # scale the image, preserving the aspect ratio
        W = img.GetWidth()
        H = img.GetHeight()
        if W > H:
            NewW = self.photo_max_size
            NewH = self.photo_max_size * H / W
        else:
            NewH = self.photo_max_size
            NewW = self.photo_max_size * W / H
        img = img.Scale(NewW,NewH)
 
        self.imageCtrl.SetBitmap(wx.BitmapFromImage(img))
        self.Refresh()
 
 
########################################################################
class QRFrame(wx.Frame):
    """"""
 
    #----------------------------------------------------------------------
    def __init__(self):
        """Constructor"""
        wx.Frame.__init__(self, None, title="QR Code Viewer", size=(550,500))
        panel = QRPanel(self)
 
if __name__ == "__main__":
    app = wx.App(False)
    frame = QRFrame()
    frame.Show()
    app.MainLoop()

The code for changing and showing the picture is explained in the previous article I wrote (and linked to above), so the only parts that you’ll probably care about are the two methods for generating the QR codes: onUseQrcode and onUsePyQR. I just took some examples from their respective websites and modified them slightly to create the QR code images. They’re very straight-forward, but not well documented, so I can’t really tell you what’s going on. Sadly at the time of this writing, the code for these projects is seriously lacking in docstrings, with only a few here and there. Still, I was able to generate some decent QR codes. The following was done using python-qrcode:

As you can see, it’s a pretty standard code. The next one is created with PyQRNative and is much denser looking:

I tried scanning both images with my Android cell phone’s barcode scanning application and both QR codes were read correctly by it. So if you’re in need of generating QR code images for your project, I hope one of these projects will fit your needs!

Source Code

• QRCodeViewer.zip

May 18, 2012 10:46 PM

We’ve been working on the Mozilla Webmaker badge system, or at least initial alpha badges for the Summer Campaign and it’s tough! We knew that going in - if it were too easy, then we probably wouldn’t end up with very valuable or robust badges - but that didn’t make it easier. There are many things to consider and it’s very easy to get caught up and stuck in the core question of what badges?  That’s a really loaded question because its not just about what to call the badges - which is a rabbit hole of itself altogether - but its also considerations around specific skills, levels and granularity (which is a huge/tough one), assessment, experience, etc. We spent days trying to answer the what badges question - should we have an HTML Level 1 and Level 2 badge, or just an HTML badge (and what do those mean?)?; should we call them Ninjas or Samurais (note: we decided on neither)?, is there a Webmaker badge that everything aggregates up to and if so what are the badges that make that up?; are all badges the same granularity?, etc. The decisions at this level are also things that more people care about and have to sign off on so that also slows down the process.

I’ve since stepped back and looked at the process and realized that there were a few considerations that actually helped us move forward - and that those considerations were one or more steps removed from the badges themselves. I’m now calling this my 3 T’s of badge system design, and so far its proving to be a helpful place to start or at least move back to when you feel you getting buried in badge level decisions. 

3 T’s of Badge System Design:

(1) Types - general categories of badges. Do you have skill badges or participation badges? Progress badges or achievement badges? To do this, you need a general understanding of the learning experiences, the community and most importantly, the goals of the badge system, but you don’t have to go super deep. You don’t, for example, need to know the exact set of skills that you want to badge. And you definitely don’t need to finalize the badge names ;). You just need to decide if you are badging skills or actions or achievements or progress, etc. Finalizing and putting some lightweight descriptions around your types of badges can really help you scope the system before diving into the questions around the specific badges that fall into each.   

Our alpha badge TYPES*:

Skill badges - I developed this skill

Participation badges - I attended or hosted this event

Achievement badges - I made this

(2) Touchpoints - next you do Touchpoints or general description of how someone will earn the TYPE of badge. This starts to pull in assessment and criteria but again, you don’t have to go super deep at first. 

Our alpha badge TOUCHPOINTS*:

Skill badges will be based on work that the learner submits, assessed by peers against a rubric. (note: this is probably even more specific than you need to go at first)

Participation badges will be based on registration for an event and proof of attendance.

Achievement badges will be based on work that the learner shares with the community.

It’s a good practice to think through if there are several possible touchpoints for each badge type (and the pros/cons of each approach). Thinking through this at the outset gives you more flexibility going into the technology considerations and helps you better work with any technology constraints you might have. For example, back up touchpoints for our badges might be:

Skill badges will be issued when the learner completes a learning challenge that cover the skills.

Participation badges will be issued by the host of an event directly to attendees.

Achievement badges will be based on completion of making exercises/projects.

(3) Technology - finally, you translate the touchpoints into high level technology requirements. 

For the first set of touchpoints, our TECHNOLOGY requirements might look something like the following:

Badges integrated into the learning tool environment and the events site

A Gallery component that learners can submit work to with a voting or rating capability for skill/achievement badges

These can be more granular but don’t have to be at this point. Just think through the basic requirements and see where you net out. It may immediately become clear that something won’t fly and you can start to work around it right away instead of way later in the process. Going back to our example, maybe we would find out that we weren’t able to have a gallery component and if this is the case, we could go back to our touchpoints and decide to use another option for those types of badges and tie those badges to the learning exercises, and thus the learning tool, instead. That decision would most likely change elements about the assessments and the specific badges we ended up defining as well, so the information flow works both ways.

What you end up with is a general map of your badge system and a basic roadmap for what needs to be built to support the badge issuing. It could also help you evaluate existing badge issuing platforms to see if they have the features that meet your needs. 

Next steps are to start to dive into each piece a bit more. Define a few of your skill badges and work through the work flow again - what is the specific touchpoint (rubric, rating required, etc.) and what are the specific features needed to support that. Press repeat, press repeat. 

Again, this is the model that we’ve accidentally started to use with our badge system design. It’s not rigorously tested by any means, just seemed to work well for our first few iterations. Would love to hear back from folks on if this is helpful, where it breaks down, etc.

-E

*Note: our alpha badges are still in alpha so are subject to change

May 18, 2012 08:40 PM

May 18, 2012 08:07 PM

Until now, we were working off of the andreasgal/b2g and andreasgal/gaia repos, but, for the sake of naming conventions, not using people's private repos and whatnot - and we now have whole new sync, build and flash processes that are much more intuitive (we hope)!

For now, the setup process for 64-bit Linux build environment remains the same; except you must also install curl for some of the config steps:

$ sudo apt-get install curl

When prompted to install any additional dependencies, accept ("y").

Cloning the new repo

$ git clone https://github.com/mozilla-b2g/B2G
 

You may want to create separate repo clones for each target for which you build.  

Before Configuring

To get a list of supported devices:
$ ./config.sh

  



 

The configure and build steps are slightly different depending on your target.

 
Configuring for Samsung Galaxy S2
$ ./config.sh galaxy-s2

Building and flashing for Samsung Galaxy S2
$ ./build.sh
$ ./flash.sh

I have also found that by default, gecko and gaia are not installed on the SGS2 for some reason.  Therefore, you must also:

$ ./flash.sh gecko
$ ./flash.sh gaia

Configuring for Nexus S
 $ ./config.sh nexus-s

Building and flashing for Nexus S 
$ ./build.sh
$ ./flash.sh 

There is more you can actually do with this; you can find detailed build instructions in the repo's readme.

May 18, 2012 07:41 PM

The Open Bastion, the subsidiary that runs events and conferences, has announced that its Open Django Central event, new to the calendar this year, is now (yes, we know) finally open to sponsors. Sponsorship is very reasonably priced, and the more money that comes in sponsorship the better we can make the event for the local Django community.

This represents a rare opportunity to connect with a diverse group of “djangstas” from the greater
Chicago and surrounding areas. The conference is targeted not just at technologists but also at
others wishing to engage with the open source world and find out what these technologies can do.

We hope that by adding your name to the roster of those supporting Open Django Central you will
indicate your enthusiastic support of and interest in open source technologies, in particular Django and Python. It's easy to ask about becoming a sponsor.

We are also still looking for speakers, so please let us know of anyone (you included, dear reader) who might be interested in giving a (technical, not marketing) talk at the conference. Slots are from 30-90 minutes.

And please act now, as time is (yes, we know) getting short!

May 18, 2012 07:31 PM

Last week I did not post one of these, because I was at Professional IT Community Conference (PICC) 2012; in New Brunswick, NJ. This is my third year in a row speaking and attending the conference, and it is always a great time. I did a session on MySQL Security (PDF slides) and another on Getting Started with a Podcast (PDF slides). There will be videos available sometime in the future, and I will link to those when they are up.

Other than that, it has been a fairly hectic two weeks. The final set of machines was moved out of the data center on Tuesday, so there was some work on that done, and there will be residual cleanup to do, but that is mostly behind us now. There was plenty of setting up machines, backing up machines and changing MySQL ACLs going on, with more to come to finally clean up all the cruft that has gone away.

And in bullet-point list format, everything else:

• We finally got all the proper drivers/firmwares/system updates on the 2 addons databases that were crashing.

• Unfortunately, now there are 2 other machines crashing (the first 2 were HP machines, these are seamicro machines).

• Removed many staging databases that were not in use, such as the one for Chocolate Factory.

• Creating new users on a database that had an application password leak.

• Lots of documentation updating happening, and some new documents created, such as what VIPs folks need to use for certain databases, instead of connecting directly. This helps make automated failover and MySQL upgrades transparent.

• Created a database and user ACLs for Tree Status

• Created a database and ACLs for the Mozilla Ignite Apps Challenge

• Created a database and ACLs for the Mozilla Popcorn website, which shows folks how to be powerful video editors.

• Created a database and ACLs for the MozTrap website.

• Data changes to Graphs

That’s all, folks!

by Sheeri at May 18, 2012 04:21 PM

In 2009, I wrote a driver to make the infra-red remote on my original MacBookAir work out-of-the-box on Linux. The driver was rejected upstream on the basis that the device would soon be supported through more generic means. In the meanwhile, it lived in Fedora's kernel tree, and I took some notes about implementing pairing, so that only your remote would work with your computer.

I'm posting this now because I wanted to poke at a MacOS X application today, and couldn't for the life of me remember the name of the program to monitor disk-activity. Hope this finds its way to a search engine near you.

• Launched the System Preferences, Security, and unlock the panel.
• In a terminal: sudo fs_usage -f filesys -w and check the output when enabling/disabling the remote.
• We can see the modified file is /Library/Preferences/com.apple.driver.AppleIRController.plist
• Installed PlistEditPro and opened the file up.
• Now try to pair a remote (menu and next together)
• You can see the UID value changing in the file. I named the remotes I had available to me:
• New remote: UID = 145 
• Old clean remote: UID = 24
• Old dirty remote: UID = 227 
• After adding some debug to the aforementioned appleir driver, in Linux, I got:
• New remote: appleir: received (5 bytes) 25 87 e0 91 02
• Old clean remote: appleir: received (5 bytes) 25 87 e0 18 03
• Old dirty remote: appleir: received (5 bytes) 25 87 e0 e3 02
• So the 4th byte is the remote's UID.

May 18, 2012 03:45 PM

Test Pilot 2 is coming soon.  In version 1, Jono had to discover what was possible.  Now that we know that everything is possible, we want to help Test Pilot instrument everything.   Here are some of our guiding principles.

easy to write studies

Here is an example study:

const observer = require("observer-service");
observer.notify("myspecialstudy",{my:'data'});

Studies message the global Observer service with observations.  Test Pilot listens.  The rest is details.

Some details:  Test Pilot 2 is a mechanism for installing addons, deciding which topics to observe, linking to external surveys, cleaning up the mess, and giving good UI and UX that ties it all together.

we meet you where you are.

No matter how you write your feature, we have you covered!  Our preferred way of running experiments is by installing Addon-SDK (née Jetpack) addons.  Of course,  your feature is so radical that it can’t be done as an addon, and only a separate binary build / Firefox branch can contain the magic.  We planned for that.

simple surveys

Linking to an external survey url is easy.  You decide who sees it, based on channel, locale, operating system and more.

the privacy you’re used to

Test Pilot remains transparent about exactly what data it collects.  We honor the contract we have with users that by sharing data about their habits, they should get something of value.  They are in charge of their data.

easier studies = more studies (and faster turnaround)

It should be cheap to do experiments.  It is always the right time to be bold.

Every feature of the browser should go in front of real users for testing.  During initial research, this might be a simple as doing hallway testing.  As a feature grows, it requires more.  Test Pilot can offer you large-scale quantitative data on “in the wild” populations, on both existing behaviours, and experimental changes to the browser interface.  Take your experiments public with Test Pilot, and find how users really react.

How can I sign up?

Need a study done?  Write to the User Research team at ( ur-team AT mozilla.com ), or stop by #ur .

Developers:  hack on the code at https://github.com/gregglind/testpilot2/.

You can join in studies today by installing Test Pilot.

by glind at May 18, 2012 02:08 PM

Visualization is an important part of any analysis. While well-designed static plots can tell rich stories about data, sometimes a little interactivity can go a long way to build intuition around a subject. Chaco is an open source plotting library that allows developers to create highly interactive plots that offer multiple perspectives on a piece [...]

May 18, 2012 12:53 PM

There’s a long road ahead to bring Europe’s “start-up darling” Berlin up to par with learning & and the web. Digital literacy and simple computational competencies are often lacking; and there’s no indication yet that Berlin schools will step to fill the gap.*

There’s an important “out of school” role to play with Berlin’s tech-savvy communities and hackerspaces, together with an existing network of media centers and educational activists.

To get closer to a vision of what this could look like, 10 educators, tech community members, and activists met on Wednesday at St. Oberholz for a community brainstorm.

The goal: to map current digital literacy needs & offerings in the city, and to scope possible next steps for a learning network in Berlin.

Connected Learning

One of theories of change driving this discussion is connected learning.

Pioneered by UC Irvine researcher Mimi Ito and the MacArthur Foundation, connected learning is about re-imagining education in the information age. It leverages today’s technologies to meet youth at their interests and passions, realized through hands-on production, shared purpose and open networks.

I personally find this model very promising, as it centers on:

• actively producing, creating, experimenting and designing
• valuing the interests of young people to steer their learning
• cross-generational collaboration
• harnessing peer culture
• linking the school, home and local community in an open network
• and honoring academic achievements.

While the steps we are taking now are small, there are a number of successful learning networks to draw inspiration and mentoring from. Among them Hive NYC and Hive Chicago, as well other models at work in Pittsburgh and other cities.

Berlin: a network for making & learning together

What could such a network look like in Berlin?

Imagine:

• Visit the Pergamon Museum and get an introduction to new methods in archeology and how to scan for objects underground.
• After unearthing a digital file of a buried statue from the museum’s learning center, you head to Open Design City, where you pick up the basics of 3D scanning and printing. You print off a copy of the statue based on the museum’s files.
• Your class had a workshop earlier that year in the Wikimedia Germany community space. So you know the basics of wiki-editing and online research. After digging through articles, you pull up an ancient inscription to go with your statue.
• Go around the corner to lasernlasern, who helps you etch the inscription into the statue using lasers.
• You’re really proud of what you made and want to tell the world. The nearest media learning center is a few minutes away. You bring your statue and some photos, and a volunteer helps you set up a blog and a gallery.
• They tell you about Coder Dojo, a youth-led initiative to learn code, which has it’s first event in Berlin next week. You sign up, eager to make a game about hunting statues and cracking ancient codes.

Needs & Offerings

At the meet-up, we mapped what we already have to offer and what we need.

It was exciting to see that collectively, we have more to offer than we have needs. Lots of important skills at the table (teaching web development, film-making, media theories, entrepreneurship, and more), as well as connections to subject-matter experts, a nation-wide network of education activists, meeting spaces, hardware, time, and even small funding to get started.

The full list is here, and please feel free to add if you something to offer to the network.

Next steps

We decided we needed to test our thinking by running an event.

An event is a concrete way to 1) try out partnerships, 2) gauge local interest, 3) experiment with the curriculum, and 4) have fun.

Together with Fabian, I’m drafting a lightweight scaffolding for a youth pop-up event this summer. Chris Lawrence from Hive NYC has written an excellent piece about how to run one of these events, from which we’ll certainly borrow many ideas.

If you’re interested in:

• Hosting a learning/hacking station (1-3hr, fun small activity that teaches a skill)
• Offering a space (large, open space holding 50-100 participants)
• Volunteering (the more, the merrier!)
• Recruiting young people (We’re old. Where do we find young people in Berlin?)
• Spreading the word

Then please join us on June 20 for a planning meeting. Location & time to be determined.

You can follow #hiveberlin for updates and also ping me (@thornet) and Fabian (@fabianmu) with ideas & questions.

• For a truly impressive example of a local school thinking in new ways, check out Evangelische Schule Berlin Zentrum.
• Images by JonLimCA

by thornet at May 18, 2012 12:17 PM

So, I've been working to release some packages today. In this case they are some eggs for Plone, which enable easier editing and creation of content types in Plone (see Products.MegamanicEditContentTypes).

I use PyPi for things we release to the community, and other customer sites that would like to use the same eggs get their buildout (zc.buildout) updated to fetch these packages.

Well, things kind of blew up today as I was updating the customer site which was using a more recent version of Plone than the one I was developing on, some renaming changes broke some of the templates so it ended up with a little bit of stress to get things working.

We have SVN where we track changes to the code so that's covered, but the release and distribution parts needs to be improved..

I see there is

http://www.python.org/dev/peps/pep-0386/

for version numbering, and that's fine.. I guess the right thing to do is to always have code eggs in an alpha status (1.0a for example) until the code has been tested on at least one different site than what it was developed on and then after a period of time, maybe 1 week of testing and bugfixing (after alpha feature freeze), create a beta (1.0b) and again after 1 week of beta testing release a release candidate (1.0rc) and after another week, create a final release (1.0). And test the code properly on the different major platform versions.

I think it's important for us at least to have a somewhat rigid testing and release system so that users of our products can depend on the quality (freedom from show-stopping bugs) of the eggs.

How do you manage these things?

May 18, 2012 11:24 AM

Here's the thirteenth installment of my ongoing series on systemd for Administrators:

Log and Service Status

This one is a short episode. One of the most commonly used commands on a systemd system is systemctl status which may be used to determine the status of a service (or other unit). It always has been a valuable tool to figure out the processes, runtime information and other meta data of a daemon running on the system.

With Fedora 17 we introduced the journal, our new logging scheme that provides structured, indexed and reliable logging on systemd systems, while providing a certain degree of compatibility with classic syslog implementations. The original reason we started to work on the journal was one specific feature idea, that to the outsider might appear simple but without the journal is difficult and inefficient to implement: along with the output of systemctl status we wanted to show the last 10 log messages of the daemon. Log data is some of the most essential bits of information we have on the status of a service. Hence it it is an obvious choice to show next to the general status of the service.

And now to make it short: at the same time as we integrated the journal into systemd and Fedora we also hooked up systemctl with it. Here's an example output:

$ systemctl status avahi-daemon.service
avahi-daemon.service - Avahi mDNS/DNS-SD Stack
	  Loaded: loaded (/usr/lib/systemd/system/avahi-daemon.service; enabled)
	  Active: active (running) since Fri, 18 May 2012 12:27:37 +0200; 14s ago
	Main PID: 8216 (avahi-daemon)
	  Status: "avahi-daemon 0.6.30 starting up."
	  CGroup: name=systemd:/system/avahi-daemon.service
		  ├ 8216 avahi-daemon: running [omega.local]
		  └ 8217 avahi-daemon: chroot helper

May 18 12:27:37 omega avahi-daemon[8216]: Joining mDNS multicast group on interface eth1.IPv4 with address 172.31.0.52.
May 18 12:27:37 omega avahi-daemon[8216]: New relevant interface eth1.IPv4 for mDNS.
May 18 12:27:37 omega avahi-daemon[8216]: Network interface enumeration completed.
May 18 12:27:37 omega avahi-daemon[8216]: Registering new address record for 192.168.122.1 on virbr0.IPv4.
May 18 12:27:37 omega avahi-daemon[8216]: Registering new address record for fd00::e269:95ff:fe87:e282 on eth1.*.
May 18 12:27:37 omega avahi-daemon[8216]: Registering new address record for 172.31.0.52 on eth1.IPv4.
May 18 12:27:37 omega avahi-daemon[8216]: Registering HINFO record with values 'X86_64'/'LINUX'.
May 18 12:27:38 omega avahi-daemon[8216]: Server startup complete. Host name is omega.local. Local service cookie is 3555095952.
May 18 12:27:38 omega avahi-daemon[8216]: Service "omega" (/services/ssh.service) successfully established.
May 18 12:27:38 omega avahi-daemon[8216]: Service "omega" (/services/sftp-ssh.service) successfully established.

This, of course, shows the status of everybody's favourite mDNS/DNS-SD daemon with a list of its processes, along with -- as promised -- the 10 most recent log lines. Mission accomplished!

There are a couple of switches available to alter the output slightly and adjust it to your needs. The two most interesting switches are -f to enable follow mode (as in tail -f) and -n to change the number of lines to show (you guessed it, as in tail -n).

The log data shown comes from three sources: everything any of the daemon's processes logged with libc's syslog() call, everything submitted using the native Journal API, plus everything any of the daemon's processes logged to STDOUT or STDERR. In short: everything the daemon generates as log data is collected, properly interleaved and shown in the same format.

And that's it already for today. It's a very simple feature, but an immensely useful one for every administrator. One of the kind "Why didn't we already do this 15 years ago?".

Stay tuned for the next installment!

May 18, 2012 10:37 AM

John MacCuish, from Mesa Analytics, pointed out that the MCS problem takes polynomial time if the graphs are planar. He writes: "Are there graphs in your likely sets, that are not planar? I have never seen a non-planar small molecule drug for example, but they may be out there. Tests for planarity are also in P. Of course it doesn't mean that solutions in P will be faster than the usual methods since N is small an the overhead for planar MCS may be large, such that N may need to be large for the planar method to beat the non-planar heuristics."

This leads to a couple of questions. One is, what is the shape of the run-time of my MCS algorithm? I don't actually know. Some tests take a very long time, but that's not enough to establish that it's polynomial for real-world compounds or exponential. I'm not going to research this now.

Another is, are their real-world small molecules which are non-planar, in the topological sense, and not the chemistry sense where all of the non-hydrogen atoms line on or near a plane?

Previous work in topologically non-planar compounds

A quick literature seach finds Synthesis of the first topologically non-planar molecule, which says:

We report here the synthesis and characterization of the tris-ether 2,5,14-trioxahexacyclo-[5.5.2.1.2^4,10.O^4,17.O^10,17]-heptadecane 3; this topologically unique (graph theory) molecule is prepared via a novel intramolecular rearrangement of either of two isomeric propellane spiro-epoxides, 1 and 2.

There's also Topological stereochemistry. 9. Synthesis and cutting "in-half" of a molecular Möbius strip by Walba, Homan, Richards, and Haltiwanger in New. J. Chem., 1993, 17, 661-681.

Quoting from the above link to Modern Physical Organic Chemistry by Ansyln and Dougherty:

We mention briefly here another topological issue that has fascinated chemists. For the overwhelming majority of organic molecules, we can draw a two-dimensional representation with no bonds crossing each other. ... It may seem surprising, but most molecules have planar graphs.

Recent efforts have produced chemical structures that successfully realize many interesting and novel topologies. A landmark was certainly the synthesis of a trefoil knot using Sauvage's Cu+/phenanthroline templating strategy.... Vögtle and co-workers have described an "all organic" approach to amide-containing trefoil knots, and have been able to separate the two enantiomeric knots using chiral chromatography. Another seminal advance in the field was the synthesis and characterization of a 'Möbius strip' molecule..."

So it's well-established that there are topologically non-planar structures. But are they in compound databases which I can access?

Searching PubChem for topologially non-planar compounds

Take a look at Scaffold Topologies II: Analysis of Chemical Databases by Wester, Pollock, Coutsias, Allu, Muresan and Oprea in PMC 2010 January 15., Published in final edited form as: J Chem Inf Model 2008 July; 48(7): 1311-1324.

Only 12 nonplanar and 2,099 spiro node topologies (all of which are planar) are present in the merged database. 9 of the nonplanar topologies are found only in PubChem and the total number of molecules represented by such topologies in the merged database is a mere 44, agreeing with Walba's assessment concerning the rarity of chemicals with nonplanar graphs.

This establishes that in 2008 there were no more than 44 topologically non-planar structures in PubChem. Can I find them?

I don't think any of the database search engines support this capability, so I need to write a program.

For that I need a method for planarity testing. Various sources say that the linear time algorithm is "widely regarded as being quite complex", but that one of the linear time planarity algorithms is part of Sage

Sage is a comprehensive mathematical software system, which uses Python. There's a pre-built binary distribution for my OS, which I downloaded. It includes Python, the IPython shell, and everything else it needs, so it really is a system and not a set of Python modules.

With Sage it's easy to make a graph and test for planarity.

% sage
----------------------------------------------------------------------
| Sage Version 5.0, Release Date: 2012-05-14                         |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
sage: d = {1: [2,3], 2: [3], 3: [4, 6], 4: [5]}
sage: g = Graph(d)
sage: g.is_planar()
True
sage: 
sage: k3_3 = {1: [2,3,4,5,6], 2: [3,4,5,6], 3: [4, 5, 6]}
sage: Graph(k3_3).is_planar()
False
sage: 

The dictionary I use as input to "Graph" contains an upper-triangle connection matrix. So to test if a molecule is topologically planar, I just need to convert its connectivity information into a dictionary of the right form, turn the dictionary into a Graph, and test the graph for is_planar().

Roadblock! My Python 2.6 modules and Sage's 2.7 Python don't mix

The prebuilt binaries include its own Python distribution, and when you use "sage", it replaces the PYTHONPATH with its own settings. This means when I run sage I don't have access to the cheminformatics tools I've already set up for my system:

% sage
----------------------------------------------------------------------
| Sage Version 5.0, Release Date: 2012-05-14                         |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
sage: import rdkit
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)

/Users/dalke/<ipython console> in <module>()

ImportError: No module named rdkit
sage: from openeye.oechem import *
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)

/Users/dalke/<ipython console> in <module>()

ImportError: No module named openeye.oechem
sage: os.environ["PYTHONPATH"]
'/Users/dalke/ftps/sage/local/lib/python'

% printenv PYTHONPATH
/Users/dalke/ftps/openeye//wrappers/v2011.Oct.1/python/:/Users/dalke/envs/RDKit_2011_12-svn
% sage
   ...
sage: import sys
sage: sys.path.insert(0, "/Users/dalke/envs/RDKit_2011_12-svn")
sage: import os
sage: os.environ["DYLD_LIBRARY_PATH"]
'/Users/dalke/ftps/sage/local/lib:/Users/dalke/ftps/sage/local/lib/R/lib::/Users/dalke/ftps/openeye//wrappers/libs:/Users/dalke/envs/RDKit_2011_12-svn/lib:/Users/dalke/ftps/sage/local/lib/R/lib'
sage: from rdkit import Chem
Fatal Python error: Interpreter not initialized (version mismatch?)

------------------------------------------------------------------------
Unhandled SIGABRT: An abort() occurred in Sage.
This probably occurred because a *compiled* component of Sage has a bug
in it and is not properly wrapped with sig_on(), sig_off(). You might
want to run Sage under gdb with 'sage -gdb' to debug this.
Sage will now terminate.
------------------------------------------------------------------------
/Users/dalke/ftps/sage/spkg/bin/sage: line 312: 56741 Abort trap              sage-ipython "$@" -i

I decided to use another technique. I would have sage call out to another program to handle the cheminformatics. For each structure it will output a line containing the identifier, the SMILES, and the needed upper-triangle dictionary data structure. One line of output will look like:

('15', 'OC1C2(C(C3C(C4(C(CC3)CC(=O)CC4)C)CC2)CC1)C', {0: [1], 1: [2, 19],
 2: [3, 20, 17], 3: [4, 18], 4: [5, 9], 5: [6, 16], 6: [7, 15, 14],
 7: [8, 10], 8: [9], 10: [11], 11: [12, 13], 13: [14], 16: [17], 18: [19]})

Even this wasn't as easy as I thought. The PYTHONPATH environment variable persists in spawned processes, which the python subprocess I started uses the wrong PYTHONPATH. I ended up setting the environment variables myself - including PYTHONHOME - before it would work correctly:

import sys
import os
env = os.environ.copy()
env["DYLD_LIBRARY_PATH"] = "/Users/dalke/ftps/openeye//wrappers/libs:/Users/dalke/envs/RDKit_2011_12-svn/lib"
env["PYTHONPATH"] = "/Users/dalke/envs/RDKit_2011_12-svn"
env["RDBASE"] = "/Users/dalke/envs/RDKit_2011_12-svn"
env["PYTHONHOME"] = "/System/Library/Frameworks/Python.framework/Versions/2.6"

import subprocess

# Import the "Graph" constructor
from sage.all import *

# Use the version of Python for which RDKit was built to use
p = subprocess.Popen(["/usr/bin/python2.6", "pubchem_connectivity.py"],
                     env = env,
                     stdout = subprocess.PIPE)
                 
for i, line in enumerate(p.stdout):
    id, smiles, d = eval(line)
    G = Graph(d)
    if not G.is_planar():
        print "======= Found one!!!"
        print id, smiles
    if i % 100 == 0:
        sys.stderr.write("%d ...\r" % (i,))
        sys.stderr.flush()

Now I just need the "pubchem_connectivity.py" program to generate the connectivity information.

Call another program to generate the connectivity information

I've been using RDKit these days, because the funding I got for the MCS project said I needed to use RDKit. I usually use OEChem, which (among other things) has much faster structure parsers than RDKit. Since I'm going to read tens of millions of structures, I wanted a way to reduce the number of structures to process.

Now, if a graph has no cycles then it's always planar. Even if it has one, two, or three cycles, it's still impossible for it to be non-planar. The number of cycles in a single component graph is E-V+1, which is the number of edges (bonds), minus the number of vertices (atoms), plus 1. So a simple test is to exclude molecules where the number of bonds is less than three greater than the number of atoms.

Mind you, this is an exclusion test which rejects graphs that cannot be planar. There are plenty of molecules with dozens or rings which are topologically planar. Also, PubChem has plenty of records with multiple components, which means I may miss a few cases. But mind you, my goal is to find some non-planar structures in PubChem, not find all of them.

I had previously converted my local copy of PubChem to a set of compressed SMILES files. A few months ago I wrote a set of Ragel definitions for SMILES, which includes a demonstration of how to use count the number of atoms and bonds in a SMILES file. I can use that unmodified as a co-process to do high-speed counting for me.

import glob
import subprocess
import gzip
import sys
from collections import defaultdict
from rdkit import Chem

# Start a co-process to count the number of atoms and bonds in a SMILES string
p = subprocess.Popen(["/Users/dalke/opensmiles-ragel/smiles_counts"],
                     stdin = subprocess.PIPE,
                     stdout = subprocess.PIPE)

filenames = glob.glob("/Users/dalke/databases/pubchem/*.smi.gz")
for i, filename in enumerate(filenames):
    msg = "Processing %d/%d\n" % (i+1, len(filenames))
    sys.stderr.write(msg)
    sys.stderr.flush()

    for line in gzip.open(filename):
        # Read a line from the data file
        smiles, id = line.split()

        # Send it to the co-process
        p.stdin.write(smiles + "\n")
        p.stdin.flush()

        # Get the counts (looks like "atoms: 34 bonds: 38")
        line = p.stdout.readline()
        _, atoms, _, bonds = line.split()
        num_atoms = int(atoms)
        num_bonds = int(bonds)

        # Skip records which cannot be non-planar
        # (Note: assumes single component structures!)
        if num_bonds < num_atoms + 3:
            continue

        # Extract the topology into upper-triangle dictionary form
        mol = Chem.MolFromSmiles(smiles)
        if mol is None:
            continue
        d = defaultdict(list)
        for bond in mol.GetBonds():
            b1 = bond.GetBeginAtomIdx()
            b2 = bond.GetEndAtomIdx()
            if b1 < b2:
                d[b1].append(b2)
            else:
                d[b2].append(b1)

        # print to stdout so the Sage program can read it
        data = (id, smiles, dict(d))
        print repr(data)

The structures!

It took many hours for my computer to chug along (while I slept). I ended up with 224 of the non-planar SMILES in the subset of 28.5 million PubChem structures I have on my machine. (In other words, bear in mind that this is not a complete search!)

Here are a few structures to show you what they look like:

The non-planar structures were a bit too verbose to show as a SMILES file. Instead, here's the full list of identifiers, which you can easily use to get the structures yourself (or follow the hyperlink to look at the non-planar depictions).

390566, 498002, 636755, 3084099, 4868274, 5104674, 6712449, 10019039, 10882690, 10895017, 10898366, 10994362, 11027681, 11126005, 11131973, 11187418, 11340053, 11350583, 11360285, 11384163, 11407796, 11672382, 14381365, 14381430, 14381432, 14381435, 16132679, 16132681, 16132995, 16132999, 16133150, 16133259, 16133262, 16133397, 16133412, 16133413, ee16133414, 16133878, 16145580, 16146229, 16146230, 16148442, 16148609, 16148632, 16148888, 16148900, 16149007, 16149114, 16149222, 16149238, 16149361, 16149362, 16149482, 16149579, 16149602, 16149827, 16149958, 16150054, 16150193, 16150399, 16150419, 16150654, 16150658, 16150833, 16150994, 16151169, 16151337, 16151360, 16151567, 16151729, 16151960, 16152362, 16152566, 16152608, 16152699, 16152729, 16153098, 16153207, 16153649, 16154275, 16154327, 16154453, 16154584, 16154971, 16155013, 16155014, 16155015, 16155069, 16155075, 16155076, 16155130, 16155140, 16155152, 16155193, 16155194, 16155197, 16155399, 16155442, 16155443, 16155456, 16155607, 16155626, 16155630, 16155631, 16155632, 16155633, 16155652, 16155784, 16155884, 16155916, 16155917, 16155920, 16155972, 16156042, 16156080, 16156082, 16156145, 16156198, 16156260, 16156261, 16156264, 16156265, 16156312, 16156411, 16156413, 16156417, 16156432, 16156495, 16156539, 16156544, 16156545, 16156609, 16156610, 16156613, 16157557, 16214951, 17749011, 21597602, 21597607, 21597610, 21597611, 21770498, 22294696, 22835058, 22835161, 22835262, 22835624, 22835636, 22835637, 23327291, 23584643, 23726086, 23727886, 23955822, 24764125, 24770227, 24770228, 24770229, 24770290, 24770291, 24871221, 24940071, 25200029, 44239114, 44303783, 44303799, 44303821, 44382489, 44397933, 44397934, 44566282, 44566284, 44575206, 44575207, 44575208, 44592641, 44592642, 44592645, 44592646, 44592647, 44592648, 44592945, 44593582, 44606373, 44606374, 46882313, 46882314, 46882315, 46882316, 46882317, 46882318, 46882319, 46882320, 46882321, 46882322, 46882323, 46882324, 46882325, 46882326, 46882327, 46882328, 46891923, 46895835, 49799159, 49799160, 49873810, 50897242, 50900298, 50900299, 50900300, 50919058, 51004304, 51026319, 52945815, 52952313, 52952314, 52952315, 52952316, 52952317, 53468167, 53468168, 53468169, 53468170, 53468171

Still, having some SMILES on-hand is nice so here are some of the shorter SMILES strings. At the least, you can use these as test-cases for an MCS search engine, and perhaps force a linear-time planar-graph-only method to fail :)

O1C23C4C5(C16OCC7C4(CC(C2(OC(=O)C3(CC5C8C6CC=C9C8(C(=O)C=CC9)C)O)C)OC7=O)C)O 390566
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cccc6 498002
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(cc6)OC 636755
[Si]123N4[Si]56N1[Si]4(N25)N36 3084099
C123C45C16C24C6C7C(C35)CC(C(C7)C)C 4868274
O1C23C4C5(C16OCC7C4(CC(C2(OC(=O)C3(CC5C8C6CC=C9C8(C(=O)C=CC9)C)O)C)OC7=O)C)O 5104674
O1C23C4C5(C16OCC7C4(CC(C2(OC(=O)C3(CC5C8C6CC=C9C8(C(=O)C=CC9)C)O)C)OC7=O)C)O 6712449
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(cc6)OC 10019039
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(c(c6)OC)OC 10882690
O1C2C34C5C(CN(C3C(C5OC)c6c(cc(c(c6)OC)O)C4C1)CC)(C(C2)O)CO 10895017
O1c2c3cccc2Cc4c5c(ccc4)Cc6c7c(ccc6)Cc8c(c(ccc8)C3)OCCOCCN(CCOCCO5)CCOCCOc9c(cccc9)OCCOCCN(CCOCC1)CCOCCO7 10898366
O1C2C34C5C(C(C2)OC(=O)C)(CN(C3C(C5OC)c6c(cc(c(c6)OC)O)C4C1)CC)COC 10994362
C123C45c6c(cccc6)C17c8c(cccc8)C2(c9c(cccc9)C3(c1c4cccc1)c1c7cccc1)c1c5cccc1 11027681
OC1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(c(c6)OC)OC 11131973
O1c2c3c(ccc2)OB4Oc5c6c(ccc5)OB1Oc7c(c(ccc7)O4)N63 11187418
C123C45C6C7C8C19C1C(C4C4C%10C22C%11C(C5C5C%12C33C(C6C5)C8CC5C9C(C2C(C35)CC%12%11)CC1%10)C4)C7 11340053
O1C2C34C5C(CN(C3C(C5OC)c6c(cc(c(c6)O)O)C4C1)CC)(C(C2)O)COC 11350583
C123C4C5C1C6C5(C4C26)C=CC#CC=CC78C9C1C7C2C1(C9C82)C=CC#CC=C3 11360285
C123C4C5C1C6C5(C4C26)C=CC#CC#CC=CC78C9C1C7C2C1(C9C82)C=CC#CC#CC=C3 11384163
O1C2C34C5C(CN(C3C(C5OC)c6c(cc(c(c6)OC)O)C4C1)CC)(C(C2)O)COC 11407796
O(c1cc2c(cc1OC)C34C56C27c8c(cc(c(c8)OC)OC)C5(c9c3cc(c(c9)OC)OC)c1c(cc(c(c1)OC)OC)C6(c1c7cc(c(c1)OC)OC)c1c4cc(c(c1)OC)OC)C 11672382
O(C12NC(=O)C3C4C56C1C7C8C59C15C6(C3C3C1C(C9C43)C#N)C2C7C5C8=O)C(=O)C 14381365
O1C2C3C45C67C2C8C3C9C42C6(C8C9=O)C3C4C7C(C5C4C2C3C#N)C1=O 14381430
O1C2C3C45C67C2C8C3C9C42C63C8C9OC(=O)C4C2C2C5C(C7C2C34)C1=O 14381432
BrC12C34C56C7(C8C9C5C5C3C9C1C8C(=O)OC1C2C2C4C(C6C2C71)OC5=O)Br 14381435
O1c2c3c4c5c6c7c3c(cc8c7c(cc6Oc9cc(ccc9)OCCOCCOc3cc1ccc3)C(=O)N(C8=O)C(CCCCCC)C)Oc1cc(ccc1)OCCOCCOc1cc(ccc1)Oc5cc1c4c(c2)C(=O)N(C1=O)C(CCCCCC)C 16214951
BrC12C34C56C7(C8C9C5C5C3C9C1C8C(=O)OC1(C2C2C4C(C6C2C71)OC5=O)Br)Br 21597602
BrC12C34C56C7C8C9C3C7C(=O)OC3C4C4C1C(=O)C(C5(C8C(C29)C(=O)OCC)Br)C4C63 21597607
BrC12C34C56C7C8C9C5C5C3C9C1C8C(=O)OC1C2C2C4C(C6C2C71)OC5=O 21597610
O1C2C3C45C67C2C8C3C9C42C6(C8C9O)C3C4C7C(C5C4C2C3C(=O)O)C1 21597611
O1C23OC(OC4C25C6C7(C1=O)C(C3OC(OC5C(=C)C4CC6)(C)C)C(CCC7)(C)C)(C)C 21770498
ClC1=CC2OC3C1C4OC5C2C4C3C5 22294696
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(cc6)OC 23327291
O1c2cc3c4cc2OCCOCCOCCOc5c(cc6c(c5)c(c7c(c6C)cc8c(c7)OCCOCCOCCOc9cc2c(cc9OCCOCCOCCO8)C4(c4c(cccc4)C32C)C)C)OCCOCCOCC1 23584643
O1C2(OCC34C5C2(CCC3OC(=O)C67C4C(OC51)CC(C6)C(=C)C7=O)C)C 23727886
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(c(c6)OC)OC 23955822
O1C2C3C45C6C1OC(C6(CCC4OC(=O)C37CC(C2)C(=C)C7=O)C)OC5 24764125
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)C6=C2CC(=C(C6)OC)OC 24871221
O1C2C34C56C(C(C2(C=C5)OC)C(O)(CCCC)C)CCN(C6Cc7c3c1c(cc7)O)CC4 44303783
O1C2C34C56C(C(C2(C=C5)OC)C(O)(CCCC)C)CN(C6Cc7c3c1c(cc7)O)CC4 44303799
O1C2C34C56C(C(C2(C=C5)OC)C(O)(CCCC)C)CCN(C6Cc7c3c1c(cc7)O)CC4 44303821
S(=O)(=O)(OCC12C3C4C(C1)CC(C3)CC4C2)N 44382489
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(c(c6)OC)OC 44592945
O1C23C4C5(C16OCC7C4(CC(C2(OC(=O)C3(CC5C8C6CC=C9C8(C(=O)C=CC9)C)O)C)OC7=O)C)O 49873810
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cccc6 50897242
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(cc6)OC 50919058
O1C23C4C5(C16OCC7C4(CC(C2(OC(=O)C3(CC5C8C6CC=C9C8(C(=O)C=CC9)C)O)C)OC7=O)C)O 51004304
O=C1N2C34N5CCC(C3CCC4(C1)C=CC5)c6c2cc(cc6)OC 51026319

Feel free to leave a comment.

May 18, 2012 08:37 AM

This article describes how to build your own rooftop awning. I recommend that if you do want an awning, purchase one from a store. They have come down in price considerably since I built mine. If you do go ahead, make sure to check with your local transportation authority and check whether such a device is legal in your country/state.

Anyway, let's start. A while ago I wanted a rooftop awning but I didn't want to pay that much money. In Australia, these awnings go for several hundred dollars, and the small ones are indeed quite small. So I built my own, instructions are below. This is how it looks like:

Sometimes it's a tough life...

• tarp with metal eyelets. IIRC mine is 2.4 x 3.0m (similar one)
• 3m 25x25mm aluminium square tube (or width of tarp otherwise)
• Two plastic caps for the square tube
• Four 1.5 inch 3/8 bolts + nuts + washers
• Four 2 inch 3/8 bolts + nuts + washers
• Four poles, one crossbar, ropes, pegs
• Wide (~5cm) velcro, 1 m is sufficient (makes 3 strips)

Tarp when rolled up for transport.

The bolt fits exactly into the groove - lucky me :)
Obscured by the rolled-up tarp is the top end of the bolt - simply a washer and a nut.

Assembly instructions

Bolt shown from the side

Crossbar mount. The bolt is horizontal when mounted.

The front bit of the rolled-up awning, taken from the other side of the car, showing the crossbar mount.

Side notes

Pros and Cons

• It's freaking awesome! Having a shady spot next to the car wherever you roll up is great!
• The awning is big. Not much of an issue once it's mounted but it's pretty unwieldy in the garage.
• The surface area covered is perfect for two people and a table, so no issues there.
• The awning is not free-standing. I believe the commercial models are, but this one needs to be pegged down. Which means that moving the car once it's set up is a no-go, you have to tear it down.
• The way I roll it results in rain collecting in the rolled-up tarp. I could roll it the other way, but that's a bit awkward. No big issue though, except after a good rain when you break at the first set of lights, a few litres of water will run down your windscreen :)
• I can't yet claim how it will hold up in a typical Queensland wind or thunderstorm. I think it will be alright, but so far I've only had it up during light winds and light rain.

May 18, 2012 07:32 AM

Today I experienced two WTF moments while trying to use computers:

1) BrowserID ended up being a total failure for me

2) Accessing people.mozilla.org is next to impossible when trying to share files across computers

I have heard great things about BrowserID, and today was my first real chance at it.  I had an account on builder.addons.mozilla.org, and this was with my <me>@mozilla.com email address.  It has been a few months since I had been on there and now it uses BrowserID for all access.  Great!!  I had signed up with BrowserID with my <me>@gmail.com address, but that failed to log me in.  So I clicked the ‘add another email address’, and got a verification email in my inbox.  Trying to verify was impossible with some cryptic error messages.  10 minutes later after trying to log in, I finally found my way to #identity and was told to try it again.  It magically worked.  OK, let me log in to my addons account, no luck.  After 15 more minutes of poking around, I found that my @mozilla.com email address worked with BrowserID just fine by testing it on another site, but it still failed on addons.

Here is my take of the problem:

• BrowserID is supposed to make logging in easier, 30 minutes of debugging and I still cannot login.
• There are no useful error and help messages on the BrowserID site, nor AMO.  How could my mom figure this out?
• Where in the world is my ‘I forgot my username/password’ link?  Honestly I could have signed up on AMO with a totally random email address and could have been wasting a lot of time.
• I found it easier to signup as a new user with a different BrowserID email, than to figure out how to login with my normal account.

My next problem occurs with accessing people.mozilla.org.  I have been using this for 3.5 years on a regular basis.  I put log files up there for people to read, zip files when I want to share some code or an build, and sometimes I create a webpage to outline data.  I depend on this as a workflow since I know of no other file server at mozilla that I can just scp files up to.  Just this past weekend, some work was done on the server and the permissions got messed up.  This was fixed, then it wasn’t, it was fixed and now it isn’t.  I can detect patterns and that is a pretty easy pattern to detect.  What really gets me is this message when I log in:

Last login: Thu May 17 18:41:20 2012 from zlb1.dmz.scl3.mozilla.com
All files stored on this server are subject to automated scans.
You shouldn’t store sensitive information on this server, and you should
avoid having production services depend on data stored here.
Files in ~/public_html may be seen by anyone on the Internet.
[jmaher@people1.dmz.scl3 ~]$

Who in their right mind would think that putting files in a folder called ‘public_html’ would not be seen by anyone on the Internet?  I expect tomorrow I will have to sign a NDA to access my people.mozilla.org account.

The big problem here is that I wasted 20 minutes doing a task that I normally do in 2 minutes and delayed getting a perma red test fixed because I couldn’t find a place to upload a fixed talos.zip to.

Enough complaining and ranting and back to work on reftests for android native!

by elvis314 at May 18, 2012 02:23 AM

When an add-on receives featured status, it assures downloads, notoriety, and a provides a healthy dose of congratulations to the developer for their hard work and good idea. Many add-on developers see becoming featured as essential to an add-ons success, whilst users get the comfort of knowing that an add-on has been reviewed and recommended by the Mozilla community.

After two successful terms of add-on advisory boards, we’re once again asking the add-on community to come forward to be part of the Featured Add-ons board; and to help pick the best consumer add-ons for the next six months.

The Featured add-ons advisory board process

For the last twelve months, volunteer board members have convened once per month to review suggestions and requests for Featured add-on consideration, highlighting some fantastic add-ons along the way. Board members must commit to submitting their own recommendations, trying out the add-ons themselves, and attending a monthly conference call to discuss suitability with the other board members.

Each Board’s term is 6 months and will be comprised of 5 community members, in addition to members of the Mozilla Add-ons team.

A note on member requirements:

Members must abstain from voting on add-ons that they have any business or personal affiliations with, as well as direct competitors of any such add-ons. Members must be picked from the add-ons community: power users, developers, and evangelists of add-ons. Priority will be given to applicants who have not served on the board before, followed by those from the first add-on board, and finally from the outgoing add-on board.

If you would like to be considered for a position on the add-ons advisory board, please email amo-featured@mozilla.org with your name and contact details, no later than May 24th, 23:59 PDT.

by dhorner at May 18, 2012 12:31 AM

After I looked at Telemetry data from users with super-slow startups, I decided to take another look at my own about:telemetry page for comparison.

First off, there were 9 SQL statements on the main thread which required longer than 100ms to execute, as well as 46 slow SQL statements on other threads. This is a bit worse than usual for my about:telemetry page + a bit surprising since my machine is a recent i7 laptop with a 7200rpm drive, running on a “maximum performance” power profile. My first suspicion fell on a busy session restore period with 5 windows and ~50 tabs but my browser was not configured to actually load the pages in the background tabs on session restore. I took some comfort in knowing that most of the components which executed those main-thread queries are already being refactored to do async I/O instead.

I was also pretty surprised by the distributions found in a few of the histograms in about:telemetry. My CACHE_DISK_SEARCH histogram, which represents the number of milliseconds to search the disk cache (grouped into buckets), had a surprisingly long tail, with 11 cache reads requiring 100ms or more:

(click for full size)

(click for full size)

  20ms            main
1856ms            createTopLevelWindow
2806ms            firstLoadURI
4025ms            delayedStartupStarted
4011ms            firstPaint
4060ms            sessionRestoreInitialized
4062ms            sessionRestoreRestoring
4604ms            delayedStartupFinished
6461ms            sessionRestored

by Vladan Djeric at May 18, 2012 12:09 AM

New release train:


CDA:

May 17, 2012 11:27 PM

TL;DR Code is eating the world the same way writing ate the world. We don't all have to be expert writers - of language or code - but we should all be literate in both.

Jett Atwood's "Please Don't Learn to Code" post shocked me a bit, because I agree with almost everything else he posts. I agree with much of it, but the "everyone should learn programming" meme is not the straw man he criticizes, it's a young nebulous movement. E.g., the Mozilla Webmakers initiative is just a series of blog posts and some communication channels organized on a wiki page. So Jeff's input could and should be formative somewhere like the next Webmakers community call.

Writing ate the world, according to Sid Meier

My MDN colleague and technical, sci-fi, and beer menu adviser Les makes a good "Please Do Learn To Code" counter-argument. I want to elaborate on his analogy to writing:

Consider writing: there's a lot to learn and it used to be a thing done only by a few scribes. But, people today get a lot of mileage out of just sticky notes and email. Sure, improving your grammar and learning how to structure an essay can help in many, many ways. But, you don't need to be a professional writer to be a professional who uses written language.

Consider the state of the world at the time writing was invented/discovered. In Civilization 5 (which is my definitive guide to history), Writing can be preceded by Pottery, Sailing, Trapping, Archery, Horseback Riding, The Wheel, Masonry, and Metal Casting. In a world of subsistence-level agricultural society, writing was a luxury knowledge skill. Maybe there's an ancient "Writing Horror: archiving and human factors" scroll telling potters, sailors, and animal trappers "Please Don't Learn to Write" - suggesting that the general populace only needs to memorize the laws of the land and sign their own name to it.

In Civ 5 - Writing enables Philosophy, which combines with Trapping to enable Civil Service, and also combines with Calendar to enable Theology, which combines with Civil Service to enable Education. Education combines with Compass to enable Astronomy & Navigation, and with Chivalry to enable Acoustics, Banking, Economics. These disciplines combine together in Scientific Theory which enables the Industrial and Modern Eras of advancement - Electricity, Telegraph, Radio, Electronics, Mass Media, Computers, Robotics, Particle Physics, Nanotechnology.

Yes, this is a simplified and gamified perspective, but the truth is, while writing is only of ultimate importance "... in the right context, for some people." the incorporation of writing into other disciplines has been the foundation of the modern world. Reading and writing are the fundamental knowledge skills for every person in every modern occupation.

Code is eating the world, according to Mark Andreessen

Writing is a technology that allows someone to externalize thoughts into a medium that can be communicated to others who parse, interpret, and process those thoughts. Code is a technology that allows someone to externalize instructions into a medium that can be communicated to computers that parse, interpret, and process those instructions. We call that externalization of code, software.

Marc Andreessen always seems to know what's going down and what's coming up. And back in December he wrote a great WSJ essay on Why Software is Eating the World - a survey of industries all across society that are being consumed by software-driven activity and operations. Code combines with book-selling to enable cloud computing. Code combines with telecom operators to enable world-wide video chat (like we were promised would happen in the 23rd or 24th century). Code combines with aviators and aerospace to launch airstrikes without putting pilots at risk. Code is eating the world, just as writing did before it. These combinations seem obvious and intuitive to those of us who make software, but they are neither obvious nor intuitive to everyone else. Jeff is spot-on when he says everyone should know enough code to know "when code might be an appropriate way to approach a problem you have" but his implication underestimates how many problems can be approached with code - i.e., almost all the problems.

What does a code-literate world look like?

Jeff has a laundry list of other problems with the "everyone should learn to code" movement. My vision of a code-literate world is quite different:

• There's less code. Coders undoubtedly realize that fully 90% of our time is spent reading code rather than writing it. When we do write code, we often refactor existing code. In a code-literate world, more people are aware of this, not fewer. So more people buying code understand that less is more.
• There's more solutions than code. Even if code hasn't eaten the problem space already, more people are aware of the ability to re-use code, especially open-source code. More people understand how and when to apply code to problems.
• There's new jobs in place of old ones. Jeff's right about entry-level programming jobs, though I dislike the trace of Silicon Valley hubris. Sure, "many people in the U.S. and around the world lack the education and skills required to participate in the great new companies coming out of the software revolution." But, "This is a tragedy since every company I work with is absolutely starved for talent. Qualified software engineers, managers, marketers and salespeople in Silicon Valley can rack up dozens of high-paying, high-upside job offers any time they want, while national unemployment and underemployment is sky high. This problem is even worse than it looks because many workers in existing industries will be stranded on the wrong side of software-based disruption and may never be able to work in their fields again. There's no way through this problem other than education, and we have a long way to go." -Andreessen

Jeff makes fun of the notion that NYC mayor Bloomberg learning to code will help him make public transit improvements. But Tulsa Web Devs are dealing with this right now as we write code for Tulsa Transit to put our local bus routes and schedules onto Google Maps. Even in the IT department, coding literacy seems low. They're "able to get around on the Internet" and they have "a basic understanding of how computers, and the Internet, work." But they buy code from Trapeze Group - they can't read it and they certainly can't write it. If the Tulsa Transit General Manager went thru Codecademy he could go to our github repository and read the code he sees. But because he doesn't know how to code, our code is as arcane to him as the box of software he buys from Trapeze. From my interactions with Code for America Brigade, I detect this scenario plays itself out in many places and in many sectors.

Code is eating the world. If people don't learn to code, they'll be eaten by it. Coding shouldn't be practiced by an elitist cabal of programmers selling software charms to illiterate masses. Our world is full of consumerism and starved for makerism already. We should encourage and teach others to code. Like writing, it creates opportunities, enables progress, and advances all of society.

by groovecoder at May 17, 2012 11:00 PM

As many of you are probably aware Dell launched Project Sputnik which is a short 6-month pet project in which Dell will take a crack at making a laptop for developers specifically those who develop Free Open Source Software.

While at UDS-Q in Oakland, CA I witnessed the awarding of three Dell XPS 13 Ultrabooks to some amazing Juju Charm authors and I also got to hold and have a look at one of the devices and one thing that I found interesting was Dell went ahead and left the Windows logo on the bottom panel of the devices they were giving away to developers and mind you the device runs Ubuntu 12.04.

Project Sputnik - Dell XPS 13 Ultrabook

I realize it is a great thing that Dell is investing money and time into developing a device that runs Ubuntu and is geared towards developers but in my mind Dell could have very easily replaced the panels before handing them off to people who work with Free Open Source Software instead of leaving the brand on the devices.

Anyways kudos to Dell for yet again giving Ubuntu a try again and hopefully we will see Dell expand their offerings to include Ubuntu laptops in the mainstream marketplace but until then there are nice vendors like System76 and ZaReason who are soaking up money from Ubuntu Users and I’m personally looking forward to CTL’s launch of a Ultrabook competitor product running Ubuntu in the near future.

by Benjamin Kerensa at May 17, 2012 10:56 PM

We are excited to announce from the Cannes Film Festival that we have selected the global winners of our 2012 Firefox Flicks competition.

Gary Kovacs, CEO of Mozilla, speaking alongside Couper Samuelson, Edward Norton and Shauna Robertson

2012 Firefox Flicks winners posing with celebrity judges and Gary Kovacs, CEO of Mozilla

After calling on creatives, filmmakers and students to help us tell the story of Mozilla Firefox, we received more than 400 submissions from thousands of filmmakers worldwide. Please see the video reel here:

Today we are awarding 25 stellar entries. For a comprehensive list of winners and runners up, please visit www.firefoxflicks.org.

Six winning films were honored today amongst Mozilla team members, judges, stars and film industry insiders at the Firefox Flicks Wrap Party at the Cannes Film Festival. The videos and the winners’ names are below. Please join us in congratulating them on their wins and their awesome work!

Be a Hero
Bogata, Columbia — Andres Felipe Mesa Rincon

Fenwick & Candy
Balmain, Australia — Gavin Fenwick Christensen, Candice Thom

I’m Falling in Love with Firefox
Cheongju-si, South Korea — Lee Sang Woo

Paranoid
London, UK – Sean O’Riordan, Remy Bazerque, Andrew Alderslade, Eleonore Cremonese

Squares
Toronto, Canada – Eric Perella, Andre Arevalo, Mark Galloway

Where the Weird Things Are Not
Alarzia, Spain –– Ferran Brooks and Ivan Cordero Raiminguez

We will now be working with the winners on plans to incorporate their work into our global marketing campaigns.

by Mozilla at May 17, 2012 10:29 PM

by Jennifer Fong at May 17, 2012 10:20 PM

This blog post is a quick explanation of how to setup clang_complete with mozilla. Note: You don’t need to build with clang to use it to complete, this means you can use it for Fennec.

Since these instruction will change from time to time I put them on the wiki page for everyone to help correct errors:

https://wiki.mozilla.org/Clang_complete_in_mozilla

by benoitgirard at May 17, 2012 10:09 PM

Discussion. Responsive images. Picture too much? Srcset weird syntax? Brevity argument. Typing hard. People lazy. Let’s go shopping?

In other, more human, words: in the wake of the current discussion about responsive images and solutions using a picture element or the srcset attribute I came across an argument that always annoys me. And I fear that the more we use that argument the more we alienate ourselves from what the web is and how it became what it is now.

It is the argument for brevity in code above everything, especially markup. Shorter is better as it means people can type more and faster and there is less opportunity for doing things wrong. I call shenanigans on this.

XHTML’s failure was not the amount of code

The argument is based on the assumption that XHTML failed because it was far too much to type and too much work. HTML5 is considered superior as we only type what we need and we can even omit a lot of “optional” code as browsers are superb and will fix our omissions for us. We write much less and it still works. We call this pragmatism. Except it isn’t. It is laziness and the arrogant assumption that we write code for browsers to execute instead of people to read.

XHTML did not fail because of the amount of things you had to write. It failed because of the redundant things you had to write, its over-complexity and that it wasn’t supported the way it was meant to in the most used browser at the time.

And even that wasn’t the issue as nobody wrote all these overly complex constructs by hand – we have editors, templates and snippets for that. Code autocompletion is quite common. We were happy adding truckloads of Object/Embed code for movies until video came around and we never typed any of that by hand. We had tools for that.

Be productively lazy

Good developers are lazy in the sense that they don’t want to repeat themselves. Instead of doing the same boring and tedious task over and over again by hand we write a script to do it for us. This is what programming is for: allow humans to do better things than the repetitive tasks computers were made for.

If you write a lot of code and it never gets used that is frustrating. Very much so. It is also pointless work. However, the mistakes of XHTML should not push us into the other extreme of writing code for computers instead of writing code that executes and is easy to understand for people who want to learn or people who will have to maintain what we wrote.

Markup is different to other code

I love markup. I love the idea of – get this – marking up a document. Adding those funky bracket things around text and URLs is not about shifting bytes, accessing a chipset or setting an interrupt. They are there to give meaning to the texts and to the URLs they contain.

Think of them as highlighting texts with a marker and writing lots of explanations in the margins explaining the meaning of the highlighted texts. Think of them as the little booklet you get with Shakespeare’s Julius Caesar explaining to you what political, social or historical tidbits the author talks about that you would never get as you don’t live in that time.

Good markup brings meaning to text. Don’t take that away from the web for the sake of brevity and technical use cases that are possibly very short-lived.

The web we all work in right now isn’t the result of writing very clever and beautiful code nor is it the result of squeezing the last byte out of our documents to make them work perfect in a certain environment. Most of the atomic micro-optimisations and performance testing and tweaking we do can be undone by a single, badly coached and still well-meaning maintainer.

The web is easy to get into – let’s start with a clean slate

The web we have right now is the result of it being the most accessible media out there. You wouldn’t know how to put your photo and a big heading with your name in the newspaper or TV. But you can learn in a few minutes flat how to write a:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Chris' page</title>
</head>
<body>
<h1>Chris rules!</h1>
<img src="http://example.com/chris.jpg" alt="Photo of Chris" 
     title="that's me, that is">
</body>
</html>

Why the doctype, the head, the charset definition and the body? Surely the browser will do that for us?

Because the code above should be the result of someone caring about the web telling you that:

• The doctype ensures predictability in displaying your page
• Defining the language means search engines have it easier to index the page and blind users don’t suffer hearing text pronounced in a different language
• The UTF-8 means that if you need international characters they will show up instead of rectangles or question marks and
• The head and body makes a clear distinction where your visible content and the instructions for the browser go.

All of this can be violated with intelligent and amazing tricks and still be valid HTML5 and cool. I am quite sure that a few people twitched at the last point and disagreed as you can style title to be visible and scripts can go in the body and there are use cases for inline styles and so on.

The point though is that none of the above hurts a web developer to write and all of it has a purpose. A structure like that makes it easier for people to learn the basics of what we do. It makes our work predictable, clean, maintainable and – at least to me – professional workmanship instead of crazy and cool geek stuff. We get tied up in the latter and one-up each other showing just what is possible and we forget that people are watching and don’t spend the time to learn the basics. Case in point? The excellent learning resource Codecademy lately added a HTML 101 course, omitting the doctype and alternative text for images in the first steps. We start to see teaching as “showing the quickest way” rather than “showing the cleanest way that explains and yields results”.

We value instant gratification over working for achieving a goal. And the gratification you feel when you achieve something you had to work for lasts longer and feels better than the fast variant. This includes making mistakes and learning from them. Giving people an environment that is incredibly forgiving as the first barrier to entry doesn’t help people grow or reach the goal on their own terms.

Fostering a new generation of webmakers

At Mozilla we have a very interesting thing going: we set a goal for ourselves to foster a community of web makers. We do workshops with journalists on how to use the web as a platform for news and entertainment. We show Popcorn as a way to produce news items that can be maintained without re-shooting. We talk to children and find playful ways to get them into making the web rather than ploughing through apps buying them, playing them for a day or so and discarding them to buy the next one. For this, we use markup and a live editor in the browser. Check out the web arcade to see what I mean.

The next generation of people coming into our market should not be virtual couch potatoes who want everything to work magically and discard it when it breaks or gets slow. Tinkering with the web is what got us where we are. Playing with the open technologies and learning from what others did made us the developers we are now. The next generation should be allowed to feel the same excitement about making that we feel now.

Be brief, but stay comprehensible

Writing not much to achieve something isn’t cool. Writing the least amount possible, getting your message across and making it easy for others to build on what you did is. It means you are free to do other, better, and – for you – much more interesting things.

Let’s focus on tools instead of muddling the basics

If you really want people to write less and achieve more, help improve and build tools for creating in a way that shortens the distance between creation and seeing the result. There is a lot of exciting work being done in this field and we need something for those who don’t want to write code. As people in the know we scoff at the Dreamweavers out there, but it is also our fault when bad code ends up on the web as we were too arrogant to help those who simply want to get their content onto the web.

by Chris at May 17, 2012 06:16 PM

After 4 solid days of learning, coding, designing, writing and generally moving the dial on their startups, WebFWD teams are back home, digesting and applying what they learned - and ate - during our Summit.

We wanted to let the teams speak for themselves:

“Our time with Jim Cook helped us realize the need to identify what we’re better at than our competitors (features, price etc.). A serendipitous meeting with Melissa and Chrissie helped us with our marketing and brand positioning. And as always, getting refueled is uber awesome when trying to do a startup. The positive energy from all of the teams is amazingly helpful. I genuinely feel like everyone there was a friend and partner in crime.” - Jaisen Maithai of OpenPhoto

“The biggest benefit to me was learning what challenges the other teams in the program were facing, and their approaches to solving them. It’s invaluable to be able to talk through and brainstorm with others that have a different perspective, but ultimately the same goal of creating a product or company with long-lasting impact.” - J Herskowitz of Tomahawk

“I had an amazing time at the summit. All the generous Mozillians that offered their expertise were so helpful, even to me, easily the least tech savvy in the group. I feel fairly confident talking about and understanding a world that just a year ago seemed completely alien. I also learned a ton from my fellow WebFWD members - being able to talk about our projects together was really helpful for me. I had a million new ideas for CASH while hashing out other people’s problems. We face so many of the same issues and other folks’ take on them is always a good way to see a problem in a new light. All in all I can safely say that Jesse and I left the summit feeling inspired and totally focused on getting CASH to the next level.” - Maggie Vail of CASH Music

“We got to meet some really incredible people with fascinating stories to tell. We learned a lot about startups and had a fun time exploring the possibilities in the open innovation space.” - Diwank Tomer of iHeartCode

“It was incredibly inspiring to be a part of this summit. Diane and Pascal have brought together people from literally all over the world who are working on some incredible projects. This really shows the power of open source development to bring people together, the backbone of what so many of us are working to do, building community and collaboration for a more open web. Entropy Wave has already benefitted from the constructively critical feedback and advice we’ve received from being a part of WebFWD, and I’m looking forward to seeing where the program goes moving forward.” - Tristan Crane of Entropy Wave

“The WebFWD summit was great accelerator for the BigBlueButton project. From the first day alone we got valuable feedback on improving our user experience, integration with WebRTC, and viable business models to support an open source project. Also, the four days of close collaboration with the WebFWD teams really built for us strong sense of fellowship between our groups as we helped each other to succeed.” - Fred Dixon of BigBlueButton

“Synbiota learned many things!

1. We learned about the culture inside of Mozilla. We were especially intrigued by the organization, efficiency, and inclusiveness of the AirMozilla Monday meet up. Very cool. These insights are already shaping our thoughts on how we will operate.
2. We learned that Mozilla is much much more than just Firefox.
3. We learned how to better analyze and break down milestones and to really understand what each milestone means and what to do if that milestone hasn’t been achieved. We are entering more and more into the data-driven mindset.
4. We learned that our project is difficult to explain and so it works best (so far) to begin a pitch with context, followed by problem, opportunity and then solution.
5. We had our first encounter with participating in an open-source, distributed hackfest. This will help us to stage future events.
6. We learned the importance of testing everyone’s products and eating one another’s dog food.
7. The WebFWD group is a global force!”

Many others participated - for giggles, you can see their responses through Diane’s Storify story….enjoy!

<noscript>[<a href="http://storify.com/tbiz/webfwd-hack-day" target="_blank">View the story &#8220;WebFWD Summit II&#8221; on Storify</a>]</noscript>

May 17, 2012 05:09 PM

The next meeting of the Python User Group Austria (PyUGAT) will take place Wednesday, May 23rd, 2012 around 6 p.m. at the Metalab in Vienna (how to get there).

Upcoming events are announced on our web site. You can also find us on Meetup.

The Agenda for this evening can be found in our wiki. This time there will be a talk about virtualenv. Bring your projects or questions to discuss with the crowd! Afterwards there is time for chit-chat and beer.

If you happen to be in or around Vienna this Wednesday, come and join us at the Metalab!

We can accommodate non German-speaking guests by switching to English, so don't be afraid to drop by!

You can also follow us on Twitter and Identica for up-to-date news. You can find logs of our previous meetings on our website.

See you on Wednesday!

May 17, 2012 04:45 PM

We have removed direct access to Java from the DOM in Gecko. Specifically, we have removed support for the ‘java’ and ‘Packages’ DOM objects. These are not part of any web standards and we don’t believe they are good for the web.

Script can still interact with Java plugin instances via NPRuntime, authors simply have to instantiate a Java plugin instance to script against.

This change will be in Firefox 15. See Mozilla bug 748343 for more information.

by Josh Aas at May 17, 2012 03:30 PM

Editor’s Note: Mozilla today announced updated user and industry adoption of Do Not Track. Check out the blog post from Alex Fowler, Mozilla Global Privacy and Policy Lead. Excerpt below.

Mozilla introduced the Do Not Track header last year to give users more control over online tracking by third parties. Since launching Do Not Track, we have seen increased industry support and user adoption of the feature. Today, we are hosting a Do Not Track event at Internet Week New York and are happy to announce new adoption statistics and industry support.

We’re excited that Twitter now supports Do Not Track and global user adoption rates continue to increase, which signifies a big step forward for Do Not Track and the Web.

Current adoption rates of Do Not Track are 8.6% for desktop users and 19% for mobile users and we see the biggest adoption rates in The Netherlands, France and the United States.*

We conducted a survey of more than 10,000 Firefox users representing 140 countries and we found some interesting results. The survey showed that 49% of users surveyed believe their privacy is respected more when Do Not Track is enabled, as opposed to only 12% who feel that way without the setting. We will share more details and specific survey results soon.

by Mozilla at May 17, 2012 03:16 PM

A few weeks ago a new Developer Preview of OS X 10.8 was released and it was discovered that as things stand now, Firefox will not run on it. With the current default settings, 10.8 will not allow any software to run unless it’s signed with an Apple Developer ID (essentially, a certificate issued by a particular Apple Root CA). We don’t know exactly when 10.8 will be released to the public but some have speculated that it could be as early as the week of June 11th at WWDC 2012. We must have a signed and released Firefox out there before the general public starts upgrading and we’ve been working hard to make that happen as soon as possible. This post will give a short history of Mac signing at Mozilla and talk about our timeline for enabling it.

Background

Code signing of Mac builds has been on our radar for a long time. Bug 400296 was originally filed in 2007. In late 2010 Syed Albiz did a ton of great work figuring out the Apple tools and how we can integrate them into our automation. That work didn’t quite get finished before his internship was completed and the bug stagnated for some time afterwards. At the start of this year there was renewed energy when Erick Dransch picked up the bug. We attempted to land his work and enable signing on nightlies in mid-April, but that ended up bouncing due to some conflicts with our upgrade to 10.7-based build machines. Erick’s internship expired before everything could be fixed up, and the bug fell to me.

After gaining access to Mozilla’s Apple Developer account on Monday there was a lot of early iteration before we got to the point where we could sign a build in a way that Mountain Lion liked. There’s multiple certificates types that one can get from Apple (“Development Certificate”, “Mac App Certificate”, “Developer ID Certificate”) and multiple versions of OS X and XCode (each with their own quirks) that one can sign with. Mostly thanks to Steven Michaud’s knowledge and assistance we figured out exactly what combination of these we’ll need to use to have signed Firefox builds that work everywhere.

Where we’re at now

At this point in time we’ve got all the tools we need to sign all Mac Firefox builds. The only blocking issue at this point is figuring out access restrictions to our Apple Developer Account, so that we can generate our final Developer ID certificates.

Endgoal

Like with Windows Authenticode signing we will have 3 different certificates for different types of Firefox builds. Dep and try builds are at the lowest level of trust and have no regular users and therefore will be signed with a self signed certificate. This means that they will not run on 10.8 unless the user has allowed “applications downloaded from anywhere” to run (which is not the default). Nightly and Aurora are at an elevated level of trust and have a userbase. These will be signed with their own Developer ID certificate. Finally, Beta and Release builds are at the highest level of trust and oversight, and represent the majority of our users. They will be signed with a separate Developer ID certificate. From a user standpoint, Nightly, Aurora, Beta and Release will all look the same but using separate certificates gives us some degree of isolation in terms of certificate revocation.

Timeline

I intend to have dep and try builds signed by the end of the week. After we figure out the access restrictions to our Developer Account we will turn on signing of Nightly builds, hopefully early/mid next week. After letting those settle for a day or two we will turn on signing of Aurora and Beta builds, hopefully by the end of next week.

If you’re interested in the technical details of signing Mac builds Erick wrote an excellent blog post detailing the trials and tribulations of writing tools around them.

by bhearsum at May 17, 2012 01:42 PM

Interestingly, I have given the presentation on MySQL and Security at least 4 times in the past 6 weeks* and it was only last night, with the sharp minds at Baron’s Central Virginia MySQL Meetup Group (sadly Baron was not there!), that someone asked about when encryption happens in the MySQL handshake.

We had been talking about how MySQL authenticates users, and how if there are no ACL’s set for a given host, MySQL will reject connections from that host – even “telnet host 3306″ will be refused – and that’s when a clever audience member asked where in the handshake process encryption started. Is it before the username is sent? Before the password is sent? Does it encrypt all traffic, even the handshake traffic?

I think that’s an excellent question, and I know there’s a few sharp minds out there who probably know the answer….otherwise I will research the answer this weekend, when I’m back home in Boston.

* Effective MySQL User Group, as part of a tutorial for Percona Live: MySQL Conference and Expo , at the Professional IT Community Conference last week, and last night at the Central Virginia MySQL Meetup Group

by Sheeri at May 17, 2012 12:50 PM

I would really like to use the same toolset for publishing POD (Print On Demand) books that I use for ebooks. Yet the pdf generation abilities of both docutils and Sphinx seem more focused on pdf “ebooks”—pdf’s that you would read from a computer—not pdf’s that are approriate to create physical books from. I think the solution is to add support for the LaTeX “memoir” package to Sphinx. I’ll eventually get around to it, but it is not on the top of my stack. To that end I’ll pay up to $200 (paypal or Amazon gift) to get that support.

For those not familiar with memoir, here’s a snippet of the README:

Memoir is a flexible class for typesetting poetry, fiction, non-fiction and mathematical works as books, reports, articles or manuscripts. Documents can use 9pt, 10pt, 11pt, 12pt, 14pt, or 17pt as the normal fontsize and, if you have scalable fonts, 20pt, 25pt, 30pt, 36pt, 48pt, or 60pt sizes, or even larger. Many methods are provided to let you create your particular design. The class incorporates over 30 of the more popular packages.

Here are the details. The first person who notifies me of two github repos that meet the following points will get $100:

• A github repo containing a Sphinx project that renders out to pdf using the memoir package. The document must have an example of:
  • Half Title Page
  • Title Page
  • Copyright Page (on verso page)
  • Dedication
  • Table of Contents
  • Introduction
  • Chapters (should start on right page)
  • An index
  • blank pages at the end
  • Customizing page size
  • The chapers should have an example of:
    • Text spanning multiple pages (lorem ipsum is fine)
    • Normal text
    • Italicized text
    • bold text
    • monospace text
    • Sections
    • footnotes
    • a code block
    • an admonition
    • a long admonition that is longer than a page
    • an admonition with a code block inside of it
    • a figure
    • a table
• A github repo containing a fork of Sphinx that has memoir support. When using this fork with the above project, a suitable pdf should fall out. Numbering of chapters is correct, page numbering is correct (odd on right side), index is correct, etc.

Ideally most of the book (Table of Contents through Index) would be pure rst (or sphinx rst++). I’m ok with raw directives on the front matter.

If you are familiar with LaTeX and Sphinx, this should be pretty straightforward. It would also be a great project for someone interesting in learning those technologies.

The person who can deliver the above will make me quite happy. But, as I said, I would pay up to $200. The remaining $100 will be appropriated to the developer (or someone else) who sees that this gets pushed upstream into Sphinx proper (ie when I see a bitbucket commit with the changes on the Sphinx project). So the person who files the bug, patch and follows through with any needed tweaks to get this into mainline will get that money. I reserve the right to split up this portion among people as well.

(I apologize in advance for the github requirement, I prefer that system, so the first deliverable needs to be there. Given that Sphinx prefers bitbucket, you need to play well with them for the second deliverable.)

May 17, 2012 12:11 PM

Scrolling performance is always important and we've made some pretty huge strides since Firefox 4. It may be helpful for me to explain some of what's happened.

In Firefox 3.6 and earlier, we had some pretty complicated code to analyze every scroll operation and figure out if we could safely scroll by just blitting some region of the window to another place and repainting the rest of the affected area. (In the very old days this was done in nsViewManager; later it was done using frame display lists.) This was quite difficult to do; for example an HTML <textarea> has a solid white background behind the scrolled text, so to prove that you can just blit the text, you have to determine that there is a solid-color background underneath the entire scrolled area. But the real problem was that it would quite often fail and we'd end up repainting the entire scrolled area, or most of it. For example, a large position:fixed element would require us to repaint the area around it. A background-attachment:fixed background would require us to repaint the entire window. This was bad because it meant scrolling sometimes "fell off a performance cliff".

In Firefox 4 we overhauled scrolling to use the new layer system. The basic idea is very simple: while scrolling, you put the content that's moving into one set of layers, and the content that's not moving in other layers. To scroll, add a translation to the transform matrix for the moving layers, repaint any parts of layers that have scrolled into view, and recomposite the window. If we do that properly, we'll never have to fall back to the "repaint everything" path; scrolling will just "always be fast". It also integrates cleanly when scrolled content uses accelerated layers for <video> and <canvas> etc.

This being the Web, there are of course a lot of complications. Separating moving from non-moving content is not easy. The infamously complex CSS z-ordering rules mean that even when scrolling a single element, you can have moving content interleaved with non-moving content so that you have to have two or more layers for each.

When you place content in these separate layers, the layers can become difficult to render. In particular text using subpixel-antialiasing placed into a moving layer, where its background is not moving, needs to be stored with separate alpha channels for each color channel ("component alpha"). This is difficult to implement efficiently. With GPU-based compositing we use shader tricks, but with CPU compositing it would be too expensive so when we encounter this situation we back off and stop caching the text in a retained layer and just draw it every time we composite the window. However even with CPU compositing, we still win on a lot of pages that use position:fixed or background-attachment:fixed.

Another thing that makes layer-accelerated scrolling difficult is when scrolled content is subject to some effect from its ancestors. The most common difficult case is when an element combines non-'visible' 'overflow' with 'border-radius' and must clip its descendants to a rounded-corner rectangle. The best way to handle this is to add support to the layer system so a container layer can clip its descendants to the rounded-rectangle border-box of an element. Until recently we didn't have that, so scrolling inside such elements was forced to repaint everything, but Nick Cameron just landed a large project to add that capability to layers and to use it in all kinds of rendering situations, including when scrolling. That means in a scrolling element that's clipping to its border-radius, the scrolled content is rendered to retained layer buffer(s) as if there was no border-radius, and then we clip to the border-radius during compositing (very efficient with a GPU since we cache the border-radius shape in a mask texture). (Nick's project provides other benefits such as accelerated video and canvas inside containers clipping to their 'border-radius' without hitting nasty fallback paths.) Summary: in Firefox 15, scrolling inside an overflow:hidden/auto element with border-radius gets a lot faster.

There is of course more to do. There are still CSS and SVG container effects we don't handle: namely filters, masks, and clip-path. Masks and filters need more layer-system support (especially filters!). Once those are done, then at least with GPU acceleration enabled it will be very difficult to hit any slow "repaint everything" paths when scrolling. (Although it's already very rare to see scrolling content inside a clip-path, mask or filter.)

The other pending important scrolling improvement is asynchronous scrolling, i.e., the ability to trigger a smooth scroll and have it happen immediately at a constant frame rate without jerkiness, even if the thread running the Web content is blocked due to Javascript execution or whatever. We've already developed most of this for Mobile Firefox (and B2G), but it needs to be made to work on desktop as well, which is not trivial. It requires enabling asynchronous compositing on all our platforms, and teaching the compositor a bit about scrolling. Once that's done, because we're able to layer-ize scrolling in almost every situation, we'll be in extremely good shape.

May 17, 2012 12:02 PM

If you tend to want to keep your music running when you log out, or control your music playing on a desktop machine from a laptop, for example, you may well use the Music Player Daemon (MPD).  If you use Ubuntu’s Unity desktop or KDE’s Plasma desktop, you may well wonder how to get the Ubuntu sound menu or Plasma’s Now Playing widget to talk to it.

Both of these use MPRIS2 to communicate with media players.  MPD, however, does not have an MPRIS2 interface.  Even if it did, it would take a bit of work to be able to use it from another computer.  What you need, then, is a “bridge” of some sort to translate.  One such possibility is mpDris2 (which, I hasten to point out, I haven’t tested).

You would run such a bridge on the computer you want to control MPD from, and point it to your MPD instance.  And voilà!  Every MPRIS2 client application that you run can talk to MPD, without knowing anything about the MPD protocol.

If you just want a headless music player on your local machine, though, you can always use Raven, which talks MPRIS2 natively.

May 17, 2012 10:45 AM

When the Thunderbird team recently started talking about how to increase the size of the Thunderbird contributor community, we thought that one interesting thing we could do was to describe the daily life of members of the Thunderbird team. In this post, we introduce readers to Thunderbird’s quality assurance and development activities, and link to profiles of Ludo, our QA Lead and David, Thunderbird Architect. In these profiles we describe the activities and experiences of each area and how contributors could help.

Can you help with beta testing?
Thunderbird QA is run by one Mozilla employee. (Click on this link to learn more about Ludovic.) Two days before the release, he tests the proposed final build of Thunderbird. He must validate three different versions of the software, testing on three kinds of mail servers and three operating systems (Windows, MacOS and Linux).

However, with a very small team and a very tight release schedule, it is a difficult challenge to make sure that we get enough testing activity in the late stage of each release. In the last while we have had a couple of releases that contained problems in a couple of specific and unusual configurations. We probably could have discovered and fixed those problems before release if we had more beta testers looking at the software before the final builds.

If you would like to help us with beta testing, just download the latest beta at and get started right away. When you launch the beta version of Thunderbird it explains how to log a bug. Don’t worry about the safety and stability of your email when testing a beta. Thunderbird is tested throughout the development cycle as features and bug fixes are integrated into the code base. All the Thunderbird team members run beta versions of Thunderbird.

Would you prefer to help with code?
During release week, Thunderbird engineers are already working on the next release, because at this point QA has already tested new features and bug fixes as they were completed. However sometimes there are surprises. (If there weren’t, we wouldn’t have to test.) When new bugs are encountered during release week, the programmers (see portrait of one of them here) kick into high gear as adrenalin levels increase, IRC channels warm up, and patches are quickly written, reviewed and integrated into the build.

While making bug fixes near the end of the release cycle is part of the game, this is not what the developers prefer to do. Most of them like to create new things – developing features, enhancing the product, fixing annoying bugs. Unfortunately, just like with QA, there are too few developers on the Thunderbird project to create all the features and enhancements we would all like to see in Thunderbird.

To address this problem, we have created a list of projects and features that we think are important but that we can’t implement because of our limited circumstances. This project is called “Up For Grabs” …

If you are interested in contributing to an open source project and know how to code, talk to the Thunderbird engineering team. Check out the Up For Grabs page to see the available projects.

by Anne-Marie Bourcier at May 17, 2012 09:53 AM

New Project Announcement:

#!/usr/bin/env python

from __future__ import print_function
import sys
from straight.command import Command, Option, SubCommand

class List(Command):
    def run_default(self, **extra):
        for line in open(self.parent.args['filename']):
            print(line.strip('\n'))

class Add(Command):
    new_todo = Option(dest='new_todo', action='append')

    def run_default(self, new_todo, **extra):
        with open(self.parent.args['filename'], 'a') as f:
            for one_todo in new_todo:
                print(one_todo.strip('\n'), file=f)

class Todo(Command):
    filename = Option(dest='filename', action='store')

    list = SubCommand('list', List)
    add = SubCommand('add', Add)

if __name__ == '__main__':
    Todo().run(sys.argv[1:])

$ ./todo.py todo.txt add "Write an example tool"
$ ./todo.py todo.txt add "Get the documentation cleaned up and on readthedocs"
$ ./todo.py todo.txt add "Blog about the project"
$ ./todo.py todo.txt list
Write an example tool
Get the documentation cleaned up and on readthedocs
Blog about the project

May 17, 2012 09:15 AM

It’s that time of the week – some great link suggestions from us in Mozilla’s Developer Engagement Team!

At the end of this blog post, you also have all the Developer Engagement team members and what they work on. If you are interested in discussing more, contributing or taking part of our work, don’t hesitate to contact us or follow us on Twitter!

Weekly links

If there is anything you think we should read or know about, don’t hesitate to post a comment, contact us on Twitter or through any other means.
The picks this week are:

• A separate mobile website: no forking way – by veteran interaction designer and content strategist Karen McGrane.
• Why Publishers Don’t Like Apps
• HTML9 Responsive Boilerstrap JS: Funny ha ha, and potentially NSFW.
• Did you ever need some knob controls in your tron-like layout? Here you go!
• Cross-Browser Debugging CSS
• LAB! – CSS 3D Earth
• iOS app success is a “lottery”: 60% (or more) of developers don’t break even
• Agile Design – Tapworthy Mobile Design
• mroth/lolcommits: Seeing is believing.

The Developer Engagement team

Mozilla’s Developer Engagement team work with writing articles, documentation – such as MDN (Mozilla Developer Network) – public speaking and generally helping and informing about open technologies and Mozilla products. If you are interested in following our work, here are the team members:

by Robert Nyman at May 17, 2012 08:53 AM

Jessica McKellar to be Keynote speaker at DjangoCon Europe!

May 17, 2012 08:37 AM

Dietrich Ayala brought this problem to my attention. For some reason Chromium browser constantly makes itself the default browser on Ubuntu 10.10 and later versions. This is annoying when you are testing Firefox Nightly Ubuntu builds. You can choose to remove the chromium-browser package, but that is not a solution.

First, I tried to reproduce this problem on multiple Ubuntu machines. To reproduce this problem, installed Firefox nightly using Firefox Nightly PPA. Remove any other browsers expect Chromium.

It took very long time to find the solution to this problem. The chromium-browser package adds /usr/local/share/applications/mimeinfo.cache which supersedes the default system application settings. The solution is edit this file and make it read-only using chattr command.

$ sudo nano /usr/local/share/applications/mimeinfo.cache 
[MIME Cache]
application/earthviewer=google-earth.desktop;
application/keyhole=google-earth.desktop;
application/vnd.google-earth.kml+xml=google-earth.desktop;
application/vnd.google-earth.kmz=google-earth.desktop;
application/xhtml_xml=google-chrome.desktop;
text/html=google-chrome.desktop;
text/xml=google-chrome.desktop;
x-scheme-handler/ftp=google-chrome.desktop;
x-scheme-handler/http=google-chrome.desktop;
x-scheme-handler/https=google-chrome.desktop;

#Remove all the google-chrome entries, like this.
[MIME Cache]
application/earthviewer=google-earth.desktop;
application/keyhole=google-earth.desktop;
application/vnd.google-earth.kml+xml=google-earth.desktop;
application/vnd.google-earth.kmz=google-earth.desktop;

# Make it read-only even for super-user
sudo chattr +i /usr/local/share/applications/mimeinfo.cache

After making these changes. Open Firefox Nightly and make it the default browser. Now this system setting will remain unchanged.

May 17, 2012 07:59 AM

We’ve been using a fork of the Fedora Project’s Mock tool to produce b2g builds for a couple months now. I described this tool in an earlier post. This post is about how we are using this fork at Mozilla.

Integrating with Buildbot automation

Mock integrates with Buildbot through the MockInit, MockInstall, MockCommand and MockProperty classes. The first two commands are used to configure and create the environment while the third and fourth are used to execute commands inside of a build environment.

MockInit is a basic wrapper for calling mock --init -r target_name. It is provided to encapsulate the logic for this relatively simple operation. Internally, initializing an environment means making sure that the environment is completely clean and has been recreated based on the contents of the root cache and the base packages specified in the mock config file.

MockInstall is also another basic wrapper. MockInstall internally calls mock -r target_name --install package1 package2 packageN. This step will install into the specified target environment the packages needed and any the packages requested depend on. This requires all build tools and dependencies be packaged and added to a Yum repository. Doing this is highly desirable regardless of whether or not Mock is being used.

MockCommand is derived from the standard ShellCommand as intended to operate like the base class. When in non-Mock mode (i.e. use_mock is set to False), this command does nothing to the configuration options and passes all calls straight to the base ShellCommand. When in Mock mode, the command runs its magic method to modify the command line to properly pass the requested buildbot environment through to the Mock call as well as passing the correctly modified command line given the working directory requested (using build environment paths) and the target that should be used.

MockProperty is to SetProperty what MockCommand is to ShellCommand with the important distinction that MockProperty inherits singly from MockCommand and duplicates logic from SetProperty.

In the current implementation, Mock has /builds/ bind mounted into the build environment. This means that all paths under /builds/ have the exact same paths in the build environment and on the build host. If the build process is substantially reworked to use a scripting framework other than Buildbot, it might make sense to stop making the /builds/ directory available inside of the build root. All commands which are “build system” commands are run under a MockCommand with use_mock set to true. Currently, there are no uploads or X11 dependent jobs run under mock. I have a blog post that details X11 access and uploads could be fixed by bind-mounting the host’s ~/.ssh directory into the build environment.

RPM and Yum infrastructure

Because all dependencies are fetched and installed through Yum+RPM on each build, it is necessary to package all tools as RPMs. How RPMs are built and Yum repositories are created is outside the scope of this document. The current setup is for Mozilla to internally mirror the host OS and the build environment OS as well as a custom repo for each distribution. Ideally, the mirrored copy of the repositories would be completely unaltered. I haven’t experimented with overriding core packages yet, but we should be fine to put newer packages that override OS packages in the custom Mozilla repository. As for building RPMs, I’ve been using spec files in my repository to build the rpms.

This repository has a directory for each distributions for which custom packages are generated, currently only ‘centos6′. Inside the distribution directory are directories for each package. There are no directories for each architecture, since rpm has facilities for specifying which architectures are valid for a given package. Each package directory contains small patches and source files needed to build the package, a tooltool manifest for larger files and a symlink to a file called “actions.sh”. This is a symlink to a script in a hidden directory at the root of the repository. In this script, there are commands for the following actions:

• fetch — use tooltool to fetch the sources
• store — add the specified file to the tooltool manifest
• build — perform a simple rpmbuild on the given spec file
• mock_build — perform a mock build of the package using upstream’s mock package

It is the responsibility of the package author to add sources, specs and manifests to the repository and upload the files to the tooltool resource host. The mock_build command can be invoked with the syntax ./actions.sh mock_build fedora-16-x86_64 x86_64 my_package.spec. The SRPMs created should be included in the list of rpm files included in Yum repository creation, since they are the canonical source of what the binary packages were built from.

Creating a new target

In the case that a new build environment configuration is needed, I’d recommend starting with an existing mock_mozilla configuration. The values that you’re likely going to be working with are the root name, arch list, base package list, bind mount and repository list. Deploying the cfg file is required and should be done through something like puppet.

Development

A release of mock_mozilla can be generated by running

git clone git://github.com/jhford/mock_mozilla.git
cd mock_mozilla
./autogen.sh
make srpm # this is to get the tarball, don't care about the srpm created here
mock --buildsrpm --spec mock_mozilla.spec --sources . --resultdir srpm --target x86_64
mock -r epel-6-x86_64 mock_mozilla-1.0.1-1.el6.src.rpm --resultdir x86_64 --target x86_64

The code to mock_mozilla is on github.

by John Ford at May 17, 2012 07:45 AM

The Lemote Yeeloong is a small notebook that is often the computer of choice for Free Software advocates, including Richard Stallman. It's powered by an 800 MHz STMicroelectronics Loongson 2F processor and has an antiquated Silicon Motion 712 graphics chip. The SM712's acceleration features are pretty subpar for today's standards, and performance of the old XFree86 Acceleration Architecture (XAA) that supports the SM712 has slowly decayed as developers move to support newer hardware and newer acceleration architectures. In short, graphics performance of the SM712 isn't very good with new X servers, so how can we improve it?

If you don't care about how pixman was optimized and just want to see the results, you can skip ahead.

pixman, the pixel-manipulation library used by cairo and X has MMX-accelerated compositing functions, written using MMX via C-level intrinsic functions, which allow the programmer to write C but still have fine-grained control over performance sensitive MMX code.

Last summer I began optimizing graphics performance of the OLPC XO-1.75 laptop. The Marvell processor it uses supports iwMMXt2, a 64-bit SIMD instruction set designed by Intel for their XScale ARM CPUs. The instruction set is predictably very similar to Intel's original MMX instruction set. By design, Intel's MMX intrinsics also support generating iwMMXt instructions, so that the same optimized C code will be easily portable to processors supporting iwMMXt. With a relatively small amount of work (as compared to writing compositing functions in ARM/iwMMXt assembly) I had pixman's MMX optimized code working on the XO-1.75 for some nice performance gains.

The Loongson 2F processor also includes a 64-bit SIMD instruction set, very similar to Intel's MMX. Its SIMD instructions use the 32 floating-point registers, and like iwMMXt it provides some useful instructions not found in x86 processors until AMD's Enhanced 3DNow! or Intel's SSE instruction sets.

So just like I did with the XO-1.75, I planned to use pixman's existing MMX code to optimize performance on my Yeeloong.

While Intel's MMX intrinsic functions are well designed, well tested, well supported, and widely used, the Loongson intrinsics are none of these. In fact, they're incomplete, badly designed, and used no where I can find (indeed, all of the instances of Loongson-optimized SIMD code I have found are written in inline assembly, which is no surprise given the state of the intrinsics). Of course, the gcc manual doesn't tell me this, so I learned it only after trying to use them with pixman.

Aside: let me pretend that I'm designing and implementing Loongson's vector intrinsics, covering an instruction set very similar to MMX, which already has an excellent set of intrinsic functions. Why would I create my own incompatible set, instead of implementing the same interface that lots of software already use?!

Using the Loongson vector intrinsics, pixman passed the test suite, and objdump verified that gcc was successfully generating vector instructions, but the performance was terrible. gcc apparently was not privy to the knowledge that the integer data types returned by the intrinsics were actually stored in floating point registers, so in between any two vector instructions you might find three or four instructions that simply copied the same data back and forth between integer and floating-point registers.

punpcklwd	$f9,$f9,$f5
    dmtc1	v0,$f8
punpcklwd	$f19,$f19,$f5
    dmfc1	t9,$f9
    dmtc1	v0,$f9
    dmtc1	t9,$f20
    dmfc1	s0,$f19
punpcklbh	$f20,$f20,$f2

This path lead no where, so I decided to take the hint from previous programmers and forget that the Loongson intrinsics exist. I still wanted to use pixman's MMX code, so I implemented Intel's MMX intrinsics myself using Loongson inline assembly. Object code size was significantly smaller and performance was better, in fact much better in some select functions, but overall was still a net loss. There must have been optimization opportunities that I was missing.

On the XO-1.75 the MMX code is faster than the generic code, so I didn't recognize inefficiencies in the MMX code the first time I worked with it, but with the Loongson it was necessary that I find and fix them. The great thing is that optimizations to this code benefit x86/MMX, ARM/iwMMXt, and Loongson.

I took a look at the book Dirty Pixels at the suggestion of pixman's maintainer, Søren Sandmann. In it, I discovered that the original MMX instruction set lacked an unsigned packed-multiply high instruction which would be useful for the over compositing operation. To work around the lack of this instruction, an extra two shifts and an add had to be used. AMD recognized this inefficiency and added the instruction in Enhanced 3DNow! and later Intel did the same with SSE. I modified the pix_multiply function to use the new instruction, and the resulting object code size shrunk by 5%.

I realized that the expand_alpha, expand_alpha_rev, and invert_colors functions that mix and duplicate pixel components could be reduced from a combined total of around 30 instruction to a single instruction each. This change further reduced object code size by another 9%.

After that, I focused on eliminating unnecessary copies to and from the vector registers. Consider this code:

__m64 vsrc = load8888 (*src);

The code loads *src into an integer register, and then load8888 loads and expands the value into a vector register. Instead, it's simpler and faster to load from memory into a vector register directly. By counting the number of dmfc1 (doubleword move from floating-point) and dmtc1 (doubleword move to floating-point) instructions I could determine which functions.

After reducing the number of unnecessary copies and adding a number of other optimizations (list available here) I was ready to see if the Yeeloong was more usable.

Results gathered from cairo's perf-trace tool confirm the real-world performance improvements given by the pixman optimizations. The image columns show the time in seconds to complete a cairo-perf-trace workload when using 32 bits per pixel and likewise image16 for 16 bits per pixel. The first column in both image and image16 groupings is the time to complete the workload without using Loongson MMI code. The second column is time to complete the workload after pixman commit c78e986085, the commit that turns on the Loongson MMI code. The third column is the time to complete the workload with pixman-0.25.6 which has many more optimizations.

As the results show, real-world performance is improved by the Loongson MMI code. I can tell a difference when using GNOME 3 (in fallback mode) on my Yeeloong.

So far this has been very successful. I've optimized pixman on an interesting platform, learned a new instruction set, and in the process found many opportunities to optimize the MMX code on x86 and ARM. I still see a bunch of things to work on with just these compositing operations alone. Beyond that, there are many other things to do like bilinear and nearest scaling functions (which are extremely important for Firefox performance). And beyond that, I've improved my understanding of pixman's code and have a few ideas for improvements in general.

Thanks to

• Danny Clark, who runs freedomincluded.com, for providing me with a Lemote Yeeloong laptop for my work on Gentoo's MIPS port.
• Søren Sandmann and Siarhei Siamashka for reviewing and helping me improve my code.

May 17, 2012 04:00 AM

One fairly common problem faced by teams at Mozilla is getting their tools plugged into the Release Engineering continuous testing infrastructure.  In the past, the workflow has been for the team to file a bug that required a Release Engineer to figure out how to build, package and deploy a new set of tools. This is not the most scalable approach.  During our recent Release Engineering workweek, we brainstormed on how to improve scaling and agility by empowering developers to help build and deploy new tools for our CI machines.

Tooltool basics

Tooltool is a client side program written in Python that uses a file manifest in concert with HTTP servers to materialize large binary payloads for use in a job.  The manifests are JSON files which list details of individual files.  Each file is represented in the JSON by a dictionary with the keys “filename”, “digest”, “size” and “algorithm”. An example is located here.  The current JSONEncoder and JSONDecoder derived classes only understand how to work with these keys. Making the JSON encoder and decoder work with an extensible version of the manifests should not be difficult.

Tooltool fetch API

When Tooltool needs to download a file from the file server it does so by creating a URL.  In the current implementation, there is a single base url that is used to construct the URLs that are to be fetched.  A good change to make would be to convert this single URL to a list of possible base urls.  The address of the file to fetch is generated using a string concatenation of the base URL, a slash, the hashing algorightm’s name, another slash and the full hash value represented in hexadecimal. Given a base url of “http://files.r.us:8080/tooltool” and a file that has a SHA512 hash value of 0123456789abcdef, Tooltool will try to fetch  “http://files.r.us:8080/tooltool/sha512/0123456789abcdef”.

There is no server component for Tooltool yet.  As a result, the file server is currently implemented as a simple directory on an HTTP host that has the correct directory structure for responding to requests.  I’ve started working on a server side component but it isn’t finished. The server side component would allow for easy uploads by developers, easy listing of contents on the server and a way to store files in a nicer way on the server’s file system.

Using Tooltool

Tooltool has four commands presently: list, validate, add and fetch.  There are global options and command arguments.  All terminal interactions after the option parser finishes is done through the Python logging API.  The default is to print logging.INFO and higher messages.  Currently, the following global options exist:

• -q/--quiet tells Tooltool to print only logging.ERROR and higher messages
• -v/--verbose specifies to print logging.INFO and higher
• -m/--manifest <file> instructs Tooltool to reference a manifest file located at the specified path
• -d/--algorithm <algorithm> instructs Tooltool to use the specified algorithm
• -o/--overwrite tells Tooltool to overwrite a local file if the filename matches the manifest but the hash value is different to the manifest
• --url specifies the base url to be used for remote operations

Listing and Validating

The two most basic commands list a manifest and validate the local files against the manifest.  The list command lists out all of the files in the manifest as well as whether they are present and/or valid.  The return code from listing is zero unless there was an error in listing the files.  Absent or invalid files will still result in an exit code of zero if there was no error in the listing process.  The validate command is used to check if all the files in the manifest are present and valid.  The exit code for validating is zero if all files in the manifest are present and their hash matches the manifest.  It is non-zero if any file is missing locally or the file does not have the same hash as the manifest.

Listing and validating a Tooltool manifest. Note the exit codes

Fetching

The fetch command is used to materialize files locally.  Before fetching a file from a remote host, Tooltool will validate local files which match the filename specified in the manifest.  The default behaviour when a local file matches the filename but not the hash value is to exit with a non-zero exit code.  If –o or –overwrite is specified on the command line Tooltool will overwrite the local file without confirmation with the remote file.  The local file will be truncated as soon as Tooltool attempts to start writing the remote file locally.

Fetching a file that exists locally with contents differing from manifest. Note the difference in behaviour when using --overwrite

Adding

Files are added to manifests with the add command. This command looks for a manifest using either the default name of manifest.tt or by the value specified by the -m/–manifest command line option. If the manifest does not exist already on the file system, a new one will be created to store the first file given. An error message will be displayed if a file is added a second time, regardless of whether or not the local contents are the same as what is in the manifest. There is currently no logic to remove a file from a manfiest or overwrite a manifest entry.

Future direction and contributions

Tooltool is a generic lookaside cache. My intention is to keep it as general as possible by not including logic to deal with payloads. We currently include a bootstrap script in the b2g manifests that understands how to take the rest of the payload and set up a working toolchain. Using a bootstrap script means that Tooltool can be tool agnostic while still allowing complex operations on the fetched tools. A standardized bootstrap script name and interface that is called by Tooltool might make sense. I’d also like to finish the server-side component that has an interface to upload files with and a method for storing files in a less obtuse method than just mirroring the API paths. A command for removing files from a manifest would be helpful, as would the ability to update a manifest with the contents of the directory as they exist right now. Another really cool feature would be the ability to configure a system wide cache directory where files are downloaded to. Once the server component is working, I’d like to add a way for servers to automatically sync their file stores when they are asked for a file that exists on another server but not locally.

Tooltool is GPLv2 and the source is available on github. The best way to contribute code is to send a pull request on github. Things are more likely to be merged if they pass the whole test suite (make check), have tests, improve Tooltool and don’t make Tooltool overly specific.

by John Ford at May 17, 2012 03:09 AM

by Jesper Kristensen at May 17, 2012 03:00 AM

by Jesper Kristensen at May 17, 2012 03:00 AM

Today, the University of Waterloo announced they’ve hired Dave Wallace to take on the role of CIO. It’s a loss for the City of Toronto, where Dave has served as CIO for the past 5 years.

I worked with Dave at City Hall, where he distinguished himself with his willingness to try new things, and a desire to go above and beyond. He helped implement the city’s 24-hour 311 call-centre, and the Toronto open data project, which now lives at Toronto.ca/open. Each time he was presented with opportunities, I found him eager to explore and listen.

When I approached him about opening up the City of Toronto’s data, he was excited along with his staff. He listened intently as Mark Surman and I pitched the idea. He came to Open Data Camp, brought his staff, and spent the day talking and listening. He put resources behind the project. He built a team around it and we worked together to implement it. He sold it to his colleagues. Without him, it wouldn’t have happened.

Action in government happens when three things come together: political leadership, public permission, and bureaucratic will. Dave not only provided the internal drive, but he shared what he learned with other municipalities and created opportunities for them to collaborate, so things like licensing and terms of service didn’t have to be written from scratch each time a new city wanted to put their data online.

In a city where public servants are accused more often than lauded, Dave’s one of the good guys. I wish him well at UW (my alma mater) — they’re lucky to have him.

Dave, you’re leaving a wonderful legacy here in Toronto. From one open data geek to another, thank you.

by rmerkley at May 17, 2012 02:49 AM

On behalf of the Jython development team, I'm pleased to announce that Jython 2.7 alpha1 is available for download. See the installation instructions.

I'd like to thank Adconion Media Group for sponsoring my work on Jython 2.7. I'd also like to thank the many contributors to Jython. Jython 2.7 alpha1 implements much of the functionality introduced by CPython 2.6 and 2.7. There are still some missing features, in particular bytearray and the io system are currently incomplete.
Please report any bugs that you find. Thanks!

May 17, 2012 01:55 AM

Mozilla is an increasingly global community.  This is important to the success of our mission. If we hope to have a world of openness and opportunity for all we should be building centers of gravity in many different locales.  Silicon Valley in California is still the center of a big chunk of the Internet industry, but Mozilla’s commitment to the Internet as a global public resource means we in particular focus on building leaders in many other places.

With this in mind my family and I have decided to get ourselves out of California for a bit.    We’re planning to move to Barcelona next September for a year.  Barcelona is not only in the heart of Europe, it’s much closer to the middle east and Africa, and it’s no further from the east coast of Latin America than California.  (Although getting to Asia may be a longer trip.)  I expect to be able to spend much more time with many more local Mozilla communities.

This is a change of geography, not of commitment to Mozilla.  I expect to spend more time meeting Mozillians and more time focusing on project  dynamics.  I want to strengthen the ability for local leaders to become regional and global leaders in Mozilla.

I also expect to spend more time representing Mozilla to governments, policy-makers and other organizations interested in Mozilla and the Internet. By being located in Europe, we will be able to give more support to the critical issues being discussed in that region.  I will also stay involved in our product efforts, as these are so key to have we achieve our mission.    Perhaps I’ll find the time to do some of the writing that would be so helpful.

I’m not (yet???) a Spanish-speaker, so I will undoubtedly spend a bunch of time off-balance and trying to figure out how basic things work.

September will be here soon.   We’re excited!

by mitchell at May 17, 2012 12:49 AM

With Hadoop, you have more flexibility in accessing files and running map-reduce jobs with java. All other languages needs to use Hadoop streaming and it feels like a second class citizen in Hadoop programming.

For those who like to write map-reduce programs in python, there are good toolkit available out there like mrjob and dumbo.
Internally, they still use Hadoop streaming to submit map-reduce jobs. These tools simplify the process of map-reduce job submission. My own experience with mrjob has been good so far. Installing and using mrjob is easy.

Installing mrjob

First ensure that you have installed a higher version of python than the default that comes with Linux (2.4.x for supporting yum). Ensure that you don't replace the existing python distribution as it breaks "yum".

Install mrjob on one of the machine in your Hadoop cluster. It is nicer to use virtualenv for creating isolated environment.

wget -O virtualenv.py http://bit.ly/virtualenv
/usr/bin/python26 virtualenv.py pythonenv
hadoopenv/bin/easy_install pip
hadoopenv/bin/pip install mrjob

May 16, 2012 09:59 PM

Jeff Atwood wrote a post on his Coding Horror blog entitled “Please Don’t Learn to Code” in which he rails against the idea that “everyone should learn programming”.

And I couldn’t agree more.

People, not everyone needs to learn programming. Only some gifted individuals (of which we professional software developers are included) need to learn programming. For the rest of you, unless you are srsly committed it will just be a meaningless chore that may damage your tiny brains.

Coding is just like surgery: if an amateur decides to code their own Angry Birds clone as a fun little project, people will literally die. Those are the stakes, folks. That’s why it should be left to those who are explicitly pursing it as a professional career.

TL; DR link

You have my assurance that I find Bloomberg’s encouragement of people to learn a technical skill personally offensive. It filled me with a rage that was only subdued after discouraging a small child from learning to play the harmonica. (What’s the kid going to do with that skill anyway? There are better ways he could spend his valuable time.)

Meanwhile, Jeff doesn’t hold back when he deals his death blow:

“Can you explain to me how Michael Bloomberg would be better at his day to day job of leading the largest city in the USA if he woke up one morning as a crack Java coder?”

Gauntlet… THROWN. The especially insidious thing (which Jeff shows that he himself is well aware of) is that even if Bloomberg’s serpentine journey into forbidden coding knowledge ended up being useless (which, yeah, maybe the mayor doesn’t need to know programming), his tweet has terrible implications for those he is in communion with. The man has 252,000 followers on Twitter. It’s as though the point of his tweet wasn’t just a casual announcement of his own 2012 resolutions, but also an encouragement for citizens to educate themselves on a technical subject.

PURE EVIL.

Think of the anarchy that would result if a fraction of them took it upon themselves to learn a new skill that is ordinarily considered not-for-average-people. They might find out that programming wasn’t as unapproachable as they previously thought!

Those who tout the “everyone can learn to code” and “coding is increasingly becoming essential” line are unaware of how preposterous their claims are. To give an example, Jeff replaces “programming” with “plumbing” in a quote from Tim O’Reilly:

Exactly! Plumbing and programming in this context are completely comparable which is why this twisted quote proves Jeff’s point. The demand for programmers isn’t high at all, and we can only expect it to decline as the 21st century progresses. And even if that wasn’t true, you don’t begin the journey to learn programming from a website like Codecademy. When has anyone ever used the Internet to learn something?

Jeff’s bullet-pointed reasons that follow his courageous, speak-truth-to-power words are at once a soothing symphony to the nerves of our exclusive software guild and a salvo against those who dare think they could learn to fish for themselves. I iterate them here with my own praising commentary:

“It assumes that more code in the world is an inherently desirable thing. [...] You should be learning to write as little code as possible. Ideally none.”

What a lot of people don’t understand is that programmers never write throwaway code. I once had the problem of wanting to download a few hundred images off of a website that had URLs like http://example.com/1.jpg, http://example.com/2.jpg, http://example.com/3.jpg, etc. up to http://example.com/347.jpg. (A local copy of an online comic, something that plenty of non-coders would want to do.)

At first I thought, “No problem. I always forget the exact function names of the networking library, but wget (a command line tool for downloading files off the web) is easy enough to use. I’ll just write a small script that writes a simple batch file that calls wget on each of these images. Something like:”

fp = open('temp.bat', 'w')
for i in range(1, 348):
    fp.write('wget http://example.com/%s.jpg\n' % (i))

Then I’d run the temp.bat file and I wouldn’t have to type out all those wget commands myself. A couple minutes later and I have all 347 images on my hard drive.

WHOA SLOW DOWN, AL. Are you sure you want to commit to writing this code? Writing a script that itself writes a script to be executed? That doesn’t sound very elegant. Isn’t that overengineering a solution to this problem? And what about supporting this code? Will it scale to millions of users? Have you thought about localization? What if I need to translate this script to Bulgarian? Do you really want to introduce this low-quality throwaway code into the world? Is that an inherently desirable thing?

Just imagine if your average layperson had the knowledge to automate simple, repetitive tasks with a quick hack like this. Then they wouldn’t have to struggle through all that manual typing and mouse clicking like a good techno-plebian. Or “ideally”, they could just give up on the problem and think there’s no practical solution (no code at all!).

“It assumes that coding is the goal. [...] Before you go rushing out to learn to code, figure out what your problem actually is.”

The core problem of this learn-coding-for-the-sake-of-coding fad is that you should have all the details of how you are going apply that skill laid out before you even write your first “Hello world” program. Having knowledge of new methods never broadens our perspectives or opens insights to novel solutions.

In my toolbox, there’s only a hammer and a saw because that’s all I need to build birdhouses. Sure, I might get ideas for new things to build if I had other tools. But I don’t know what those things are right now, so why do I need anything beyond what I already have? And hopefully if I stick to what I know, I’ll never find out.

“Software developers tend to be software addicts who think their job is to write code. But it’s not. Their job is to solve problems. Don’t celebrate the creation of code, celebrate the creation of solutions.”

Totally on the ball. It’s just like when one of my friends wanted to start an afterschool music program. “Hey now, the world doesn’t need another afterschool program. It needs an effective solution to lower drop-out rates, increase test scores, and prevent gang violence.” Thankfully, I was able to convince him to can the idea.

“It assumes that adding naive, novice, not-even-sure-they-like-this-whole-programming-thing coders to the workforce is a net positive for the world.”

I’ve seen some terrible code in my time, and the thought of these novice programmers being hired and putting their code on the market send shivers down my spine. Maybe if there was some way that companies could judge which candidates were qualified or not before hiring them (like a piece of paper listing their previous experience and education, or some sort of one-on-one conversation about their domain knowledge) it would be okay for people with less-than-absolute-mastery skill in coding to exist. Sadly, I can think of nothing that would prevent crap programmers from instantly being put in charge of the software for nuclear power plants.

You don’t create an effective workforce of software developers by encouraging many people to try their hand at coding and see who sticks with it. Rather, you need to scare off anyone who isn’t “serious” about coding. That’s the attitude that has led to the arrogant, petty, sarcastic, satire-writing generation of computer programmers I proudly say we have today.

“It implies that there’s a thin, easily permeable membrane between learning to program and getting paid to program professionally.”

The cruelest joke is played on the ones who take on the task of learning to program. Jeff specifically cites the book “Teach Yourself Perl in 24 Hours” as contributing to this misleading atmosphere. How could anyone not realize that this claim was a gimmick based on the fact that the book had 24 chapters? When I was first starting to learn Perl, I literally thought this $35 book and a single day would be all I needed to become an employed Perl developer who knew every facet of the language. Surely I can’t be the only one who thought this. I hadn’t been so disappointed since the day I saw “The NeverEnding Story” and ended up leaving the movie theater a mere two hours later.

In Conclusion

Why would the average person need to learn programming? When would they ever use that knowledge if they weren’t going to become a software engineer? Just like mathematics, a musical instrument, a foreign language, all sports ever invented, cooking, dancing, knitting, sailing, and everything beyond a 3rd grade education, you can get by in life just fine without it. I mean, when have you ever heard of someone enjoying programming just as a hobby and creative pursuit? When has anyone said that programming is great way to improve general analytical skills?

Remember, Jeff isn’t just saying that most professions and lifestyles wouldn’t be significantly enhanced by programming ability. He isn’t just saying that “coding is the new literacy that you have to have” is hyperbole. (Both are arguable points.) If he was, he would have titled his post, “You Don’t Actually Need to Learn to Code”. Rather, he wants to keep the unwashed masses from embarrassing themselves with their amateur code which he and the other elite coders will end up having to debug. That’s why the title is “Please Don’t Learn to Code”.

It’s his plea for you to not even try.

Folks, programming is a privilege and a responsibility and not everyone should attempt to have it. In that way, it’s just like writing. If an army of amateurs took up keyboards to articulate their thoughts, just imagine what kind of dopey, smug opinions would be posted to the Internet.

Signed,
A Concerned Rock Star Programmer

May 16, 2012 08:59 PM

European-based identity and security analyst firm, KuppingerCole, announced last week that OpenID Connect was awarded the 2012 European Identity and Cloud Award in the category for Best Innovation/New Standard. This recognition was largely based on OpenID Connect’s potential to significantly change digital identity using a simple interoperable Internet identity protocol to improve the way we interact with each other online.

According to Dave Kearns of KuppingerCole, OpenID Connect’s design philosophy to “make simple things simple and make complicated things possible” can play a critical role in creating the technical specifications (“tools”) necessary for advancing Internet identity across both traditional and evolving digital platforms.

“What’s most impressive is that this elegantly simple design resulted from the cooperation of such a diverse global set of contributors. I expect OpenID Connect to have a substantial positive impact on usable, secure identity solutions both for traditional computing platforms and mobile devices.”

As Dave sees it, OpenID Connect is to OpenID 2.0 as Gigabit Ethernet is to Bob Metcalfe’s original Ethernet. First, where integration of OpenID 2.0 requires an extension, OpenID Connect, which performs many of the same functions as OpenID 2.0, is built into the protocol and is API-friendly. Second, OpenID Connect provides a secure, flexible and interoperable identity layer on top of OAuth 2.0 specifications, enabling participants to exchange any claims relevant to their application. It doesn’t define ways to authenticate users or communicate information about them. Instead, OpenID Connect uses a default set of common claims about a user (e.g., name, email address, user identifier enabling SSO) to allow digital identities to be used across websites and applications.

In an indirect but important way, OpenID Connect supports the mission of the Open Identity Exchange (OIX), which similarly suggests open source for Internet identities. The relationships, dependencies and synergies between OpenID Connect and the OIX can play an integral role in the advancement of digital identities.

OpenID Connect’s modular design “tools” give relying parties the flexibility to deploy the attributes they need to improve operational efficiency and security while remaining interoperable. From a policy standpoint, OIX helps set the stage for industry stakeholders and policymakers to create and publish the policy “rules” for open identity trust frameworks that improve the user experience and protect identity and privacy..

Together, this new open approach for creating custom “tools and rules” can play a useful role in establishing the levels of assurance and elevating trust in internet identities across multiple jurisdictions and improving the way public and private industry interacts with users over the Internet.

May 16, 2012 08:39 PM

There is now a link in the footer of www.mozilla.org that invites people to contribute to that page.

We were originally thinking of this as a way to reach out to webdev volunteers, but it became clear that there are many others ways to contribute to a page including translating, designing, writing and testing.

Inviting people to participate on every page could be very powerful. Many people don’t know they can contribute to Mozilla so they wouldn’t think to look for our Get Involved page.

We’ll be able to use the Get Involved dashboard that the Metrics team recently created to see if this help us connect with more potential volunteers. If so, we could look at adding this invitation to participate on every page of every site in the Mozilla universe.

by davidwboswell at May 16, 2012 08:33 PM

No meeting on Wednesday, May 16, 2012.  See you next month.

May 16, 2012 08:29 PM

Sometimes you have an optimized build for whatever reason (say you’re doing a lot of profiling) but optimizations make non trivial debugging impossible. You don’t have an up to date build without optimization so you whine, start a non optimize build and start looking at bugzilla for 20 mins.

Frankenstein optimized/non optimized build to the rescue! Simply add:

CXXFLAGS += -O0 -g

to the Makefile for the module(s) you’re interested in debugging, for me it was gfx/layers/Makefile.in.

How does this work? Well optimizations are done at the object level and each object file are built to follow the ABI. As long as the ABI is followed, and it really really should, then you can expect this to work without any problems.

Disclaimer: This isn’t supported! If you have problems then do a clobber build.

by benoitgirard at May 16, 2012 06:13 PM

Every week I post an overview on what’s been happening in the mobile (browsing) world and is relevant to Mozilla.

• Updated Android version distribution numbers are available
• Google’s Motorola acquisition expected to be completed soon
• Android 5.0, code-named Jelly Bean, rumoured to be released in fall with multiple flagship devices
• Baidu to be releasing Yi-based devices
• 56.1% of smartphone shipments in Q1 featured Android
• Samsung confirmed as top mobile phone vendor by shipments in Q1

Updated Android version distribution numbers put Gingerbread at 64% of the market, with API level 9 at 0.5% and level 10 at 63.9%, Froyo (level 8) at 20.8% and Ice Cream Sandwich at almost 5%, with most of it on API level 15.

Google declared that it expects its acquisition of Motorola to be completed soon, before the first half of the year. After having passed regulatory approval in the US and the US, the company is now waiting from go-ahead from the authorities in China. The closing of the deal will likely raise more concerns over competition inside the Android ecosystem.

Details about the upcoming version of Android, version 5.0 code-named Jelly Bean, have emerged.The launch is rumoured to happen before Thanksgiving and to feature more OEMs that will produce the version’s flagship devices. It is to be expected that they will also be featured for sale in Google’s Play Store.

Baidu, China’s incumbent search engine, is set to release a series of new mobile devices based on its Yi mobile platform, a fork of Android, that was announced in September 2011. Dell is reported to be the company’s hardware partner for this venture in a country which recently surpassed the US as the fastest growing market for new Android & iOs activations.

The smartphone platform market was further dominated by Android in Q1 smartphone shipments, where Google’s ecosystem captured 56.1% of the market with 81 million units, up from 36.4% of the market and 36.3 million units in the same period of 2011. Apple’s platform was on the second place with 33 million and 22.9% of the market, also up from 16 million and 16.9% marketshare in Q1 of 2011. Other platforms with significant shares are Symbian with 8.6% of the market, followed by BB OS with 6.9%, Bada with 2.7% and Windows Mobile and Windows Phone 7 which together account for 1.9%.

Q1 results of phone and smartphone shipments are out, confirming Samsung as having taken the lead as the top mobile phone vendor with 86 million units and 20.7% of the market, up from 68 million and 16.1% of the market the previous year. On second place there is Nokia, with 83 million and 19.8% marketshare, which is on a downward trend from its Q1 2011 result of 107 million corresponding to a 25.1% marketshare. Apple takes 3rd place with 33 million and 7.9% of the market, up from 16 million and 3.9%. Further down the top there are ZTE, LG, Huawei, RIM, Motorola, Sony and HTC with significant shares of the market.

by Irina Sandu at May 16, 2012 05:41 PM

Building A Great Work Culture (Inspire, Empower And Collaborate)

May 16, 2012 05:37 PM

“I’m a professional product manager and if you need help I would be delighted to join.” — from message posted on Get Involved page

Are you interested in helping people get involved with Mozilla, like this person who wants to help with product management? Then join us to discuss community building at Mozilla.

• Meeting time: Thursday, May 17 at 10:00 AM Pacific
• Topics: Localizing Get Involved page and increasing community building skills with Mozilla Brain Builders
• Video: David Boswell’s vidyo room (link for guest access is on agenda)
• Audio: 1-800-707-2533, pin: 369, conf: 9634#
• Room: Get to do Choppa in Mountain View 3rd floor
• Back channel: irc.mozilla.org#mozillians

Note that the video and audio information has changed from previous meetings.

If you have a question you’d like to ask the group, please feel free to edit the agenda on the wiki.

by davidwboswell at May 16, 2012 05:17 PM

I’m pleased to report some user visible progress from the performance and metrics work week. Sign in to the Telemetry dashboard now uses Persona (aka, BrowserID).

No special permission is required. The Telemetry dashboard is open to all. Don’t have a Persona account? No problem. Click the sign in button to be prompted to create a free account.

This change is now live. You can try it yourself by visiting the Telemetry dashboard at http://mzl.la/telemetrydash.

by Lawrence Mandel at May 16, 2012 05:03 PM

Berlin-based WebFWD Scout Joseph Somogyi is keeping busy. He will present WebFWD next week on May 25 at LinuxTag, the biggest Linux and Open Source exhibition in Europe, full of workshops, camps and many conference tracks running in parallel. Linux Tag, a 4-day government-sponsored event for every kind of Open Source, is expecting a host of exhibitors and attendees.

Joseph also found time recently to share a pointer to The Architecture of Open Source Applications, Vol I-II (2012), edited by Amy Brown and Greg Wilson, and available online in its entirety. Chapters are written by a collection of open source heroes, including one called Firefox Release Engineering, by Mozillians Chris AtLee, Lukas Blakk, John O’Duinn, and Armen Zambrano Gasparnian.

Joseph quotes:

“Architects look at thousands of buildings during their training, and study critiques of those buildings written by masters. In contrast, most software developers only ever get to know a handful of large programs well—usually programs they wrote themselves—and never study the great programs of history. As a result, they repeat one another’s mistakes rather than building on one another’s successes.

Our goal is to change that. In these two books, the authors of four dozen open source applications explain how their software is structured, and why. What are each program’s major components? How do they interact? And what did their builders learn during their development? In answering these questions, the contributors to these books provide unique insights into how they think.”

Other sightings: These closing links come from startup country, Silicon Valley, where a couple high profile acquisition plays triggered plenty of noise and some thoughtful postings earlier this spring:

• High Scalability - The Instagram Architecture Facebook Bought for a Cool Billion Dollars
• Learning from Slideshare’s Founders - NextWala

May 16, 2012 03:04 PM

We actually wanted to enabled Incremental GC on Nightly, but again we had some fallout and it had to be backed out again. Bill thinks it should reland at the end of the week.

We are happy to welcome Benjamin Peterson, who is going to join us this summer as an intern working on SpiderMonkey’s ES6 support. Benjamin is an active python contributor. He has already started implementing rest parameters.

Till Schneidereit, (a fellow German, finally!) started picking up some GC related bugs, thank you and feel welcome.

In an effort to reduce the memory usage of average JavaScript applications (MemShrink \o/), we came to the conclusion that it is okay to throw away JIT code compiled by Jäger on every Garbage Collection run. Unfortunately this doesn’t work very well for animation heavy scripts like games, where recompiling would introduce long pauses. Brian fixed that.

Jason showed us how to use the new Debugger API to debug JavaScript code running in Firefox.

David Mandelin and me blogged about the SpiderMonkey API (JSAPI), and what needs to change, C++ yeah!

The DataView object landed, thanks to the work of Steve.

Luke just finished a patch that is going to speed up the handling of some function parameters/variables. Besides blocking more IonMonkey performance improvements, it already showed 10% better scores on the v8 early-boyer benchmark. (Bug 659577)

Jan has been working on chunked compilation which should help IonMonkey with very large scripts. But because this is a very broad change and the Ion team likes to focus on stabilizing, fixing crashes and test failures first, this is going to land after the initial release. Luckily these kind of large scripts are uncommon for normal JavaScript, but they are often found in Emscripten compiled code. JägerMonkey (+TI) which has chunked compilation is still going to help those scripts.

Edit: Republished because of some tumblr problems.

May 16, 2012 01:13 PM

TL;DR: I think coding is an essential skill for modern humans surrounded by code and machines that run it. Please learn to code.

I disagree with Jeff Atwood on "Please don't learn to code":

The "everyone should learn to code" movement isn't just wrong because it falsely equates coding with essential life skills like reading, writing, and math. I wish. It is wrong in so many other ways.

In fact, I do regard coding as an essential modern skill. Yes, right alongside reading, writing, and 'rithmatic. At least this part of the post had me nodding:

I suppose I can support learning a tiny bit about programming just so you can recognize what code is, and when code might be an appropriate way to approach a problem you have. But I can also recognize plumbing problems when I see them without any particular training in the area.

Visible pipes

Luckily, pipes are not as occult as code. If you go into a basement or open the door under a sink, you can see them and follow where they go. That's some training, albeit informal or self-directed.

I'm not sure how you'd get exposed to code in the same way: View Source in a browser used to be a decent start on the web, but that's less helpful lately. Code elsewhere has typically been hard to get at, Open Source notwithstanding.

Still, I'd bet there are people in the world for whom running water comes from magic and drains into magic. Where magic means: "I never thought about it, never needed to, and am sometimes vaguely afraid of it."

For that class of homeowner, the kitchen floods when the sink springs a leak, until an expert arrives. It's not the end of the world, and plenty of people get by just fine like that. But, I certainly wouldn't agree that "Please don't learn about pipes" is good advice in general.

Learning by doing

Admittedly, "learn to plumb" isn't the same as "learn about pipes". But, is there a difference between "learn to code" and "learn about code"? I don't think so. Like writing, code doesn't seem like a thing that's easy to learn about without doing it.

When I write "coder", I generally mean this: Someone who is capable of encoding his or her intent and decision process into a form that can drive a CPU to perform tasks on his or her behalf.

That's a very broad definition, but it implies a lot. First, you have to realize that you can make a CPU can do things on your behalf--it's okay, you won't break it. It's a tool made by humans and you as a human can understand it. Then, you need a notion of algorithmic thinking, in order to formulate your intent and reasoning in a form that a CPU can execute. These are not natural or intuitive things.

I agree with that it's good to "recognize what code is, and when code might be an appropriate way to approach a problem". But, if you've never made a CPU do your bidding, it's easy to see it as a mysterious black box with a will of its own--possibly malicious or at least capricious.

And, if you've never worked to force your thoughts into to the confoundingly literal and common senseless constraints of computer programming, it's hard to even imagine how code works. If you can't imagine how it works, how do you work it into your mental toolkit for getting things done?

Learn to code, and a lot of other things get dragged into your head along the way.

Who needs all these coders?

And then, there were these bits from "Please Don't Learn to Code":

It assumes that adding naive, novice, not-even-sure-they-like-this-whole-programming-thing coders to the workforce is a net positive for the world.

It implies that there's a thin, easily permeable membrane between learning to program and getting paid to program professionally.

This is looking at work from the wrong angle. It isn't about getting paid to program, so much as coding to be good at your job.

I'm not talking about Java-heads who live all day in Eclipse. I'm talking about the Excel-head who used to rock VBA macros, who maybe just started playing with Google Apps Script. I have no idea how popular Google Apps Script might or might not be, but I've seen some crazy amazing things done in VBA by sales and account reps who'd punch you in the nose if you called them geeks.

As far as I can tell, the future of work is heading toward more work with greater volumes of information and data. Should professional programers be the only people in an organization who know how to apply computational power to solve problems? Maybe the vendors will sweep in, clean up all the cybercrud, and get the real work done for us.

Programming should not be a priesthood

Consider writing: there's a lot to learn and it used to be a thing done only by a few scribes. But, people today get a lot of mileage out of just sticky notes and email. Sure, improving your grammar and learning how to structure an essay can help in many, many ways. But, you don't need to be a professional writer to be a professional who uses written language.

The same can apply for coding. The problem, though, is that the sticky-notes-and-email level of competence barely exists or is near impossible to access. So, not only do I think we need more coders--we also need more tools that support coding and make coding more accessible. I think we should support professionals who use code.

More than that, I think we should encourage and support humans who code. I really do consider coding next to reading, writing, and math. I don't think we can all rely on someone else to write the perfect app for the work we'll need to do in the future. I expect the successful people will be those who can apply Taco Bell programming to reams of data and find answers. We'll need to ride bicycles, not tricycles.

May 16, 2012 01:00 PM

“thinkinsights” and Google have released some fascinating data about smartphone usage, gathered from detailed consumer surveys.

All the presentation reports (right hand column) have roughly the same format. Why not download the report for your country, and the one for Brazil (the launch country for B2G) and see how different things are there compared to where you live?

Compared to the UK, in Brazil:

• Far fewer people have smartphones (14% vs 51%)
• Smartphones are used less often (40% vs 59% daily)
• They are used on-the-go less often (64% vs 86%) – poorer network coverage?
• Social networking is proportionately more popular than email
• Smartphone gaming is significantly less popular (39% vs 62%)
• They use fewer apps, even free ones (14 vs 23 installed)
• Almost all smartphone users are urban
• Cohabitation is significantly less common (20% vs 11%)

Thoughts: reading between the lines, network coverage is poorer and data-on-the-go is harder to find. We need to make sure B2G phones and apps are solid in absence of a good network connection. Also, the phone will be the only computing device for many users.

by gerv at May 16, 2012 12:04 PM

Mozilla zal op 31 mei met een stand aanwezig zijn bij het 'Open Overheid Congres' in Utrecht. Hieronder een samenvatting van de inhoud van dit congres. Meer informatie kunt u vinden op http://www.ecp-epn.nl/open-overheid-congres. Het congres is gratis toegankelijk, dus kom langs en laat u bijpraten over openheid en bezoek tussendoor de stands van open-source leveranciers als Mozilla en anderen.

Open standaarden, Open source software en Open data: drie
losse onderwerpen die in de praktijk enorm verweven zijn, nu in één congres! Kom op 31 mei naar het Open Overheid Congres, georganiseerd door ECP, Forum Standaardisatie, ICTU, VKA, TNO en OSSLO.

Laat u bijpraten over de stand van zaken en ontmoet uitvoerders van beleid, beleidsmakers, IT-verantwoordelijken van overheidsorganisaties en marktpartijen. Vernieuwende denkrichtingen en praktijkvoorbeelden zullen u inspireren!

Open Overheid congres 
Donderdag 31 mei 2012
12.00 – 17.00 uur (incl. lunch)
Beatrix Theater te Utrecht

May 16, 2012 10:27 AM

An under-3-minute video short story by Tom Scott. “A science fiction story about what you see when you die. Or: the Singularity, ruined by lawyers.”

by gerv at May 16, 2012 08:05 AM

Add-ons

The main news in the past two weeks has been about Kyle Huey’s patch that prevents most chrome-to-content leaks, which are the most common kind of add-on leak.  Testing showed it worked beautifully, but caused a knock-on leak in add-ons built with old versions (1.3 and earlier) of the Add-on SDK.  (This received a lot of attention.)  Kyle then made a slight tweak that fixed that knock-on leak.  So we’re currently still on track for Firefox 15 being “the one that fixes add-on leaks”.

For completeness, here are the add-ons that we know were temporarily affected by that knock-on leak:  Wallflower, Visual Hashing, Translate This!, Easy YouTube Video Downloader.  They (and probably quite a few others) are all working fine again now.

Here’s a quote from an email that one user sent to Kyle this week.

Firefox was leaking about 1.5GB per hour for me. It started with Firefox 3. I tracked it down to Ghostery and NoScript, but even without those addons it leaked about 500MB per hour of browsing.

GC and CC times got up into the 10 second range. Ugly. Really really ugly! And this is on top of the line massively overclocked hardware, too. I had to install a new addon to add a restart button to Firefox, because Firefox froze solid after hitting 2GB of memory usage. I also patched it after every update to allow up to 4GB, buying a little more time…

Then your patch comes along and solves it all… you are awesome man – totally awesome!

Another user — one who uses the leaky Autopager add-on — commented on Kyle’s blog.

Certainly, before this fix I would find that Firefox often became sluggish (input lag, slow paint operations, less than silky smooth scroll animations) as the memory usage built up. It’s hard to say how much various factors contributed to the whole, but GC pauses did undoubtedly cause the scroll animation stuttering.

Restarting was the cure. I haven’t noticed the same symptoms since, and while I haven’t had enough chance to make a conclusive judgement, the signs certainly seem to be good.

I have a full tab strip more often than not, and Fx set to load tabs from last time. This is offset by the wonderful and elegantly simple tabs on demand feature. I’m running a 2 year old laptop with 4GB ram.

And while we’re on the topic, here’s a comment from my blog.

Opened my firefox today, 30+ Tabs (only counting the ones in the active group, the others aren’t loaded), using just little more than 330 MB of RAM. A year ago, with Firefox 4, this would have been impossible. Keep it going!

Good times.

The following add-ons had zombie compartments fixed:  Youtube Ratings Preview, SPDY Indicator  It’s likely these leaks would have been fixed by Kyle’s change, but since Firefox 15 won’t be released until August 28, it’s good that they’ve been fixed now.  (Indeed, the AMO review policy still requires that add-ons not cause zombie compartments with the current release of Firefox;  that policy may be revisited once Firefox 15 is released.)

Compartment-per-global

The other big news is that compartment-per-global (CPG) landed, thanks to the work of various people, especially Luke Wagner and Bobby Holley.  Bobby explained what this means and explored some of the consequences.

CPG will allow lots of things within Firefox to become simpler and faster.  The main disadvantage is, unfortunately, increased memory consumption, as can be seen on areweslimyet.com.  (Thanks to Luke, this increase was less than it could have been.)  This is mostly due to more fragmentation in the JavaScript heap — we now have many more compartments, and each 4KB heap arena cannot be shared between compartments, so there are many more partially empty arenas present.

You might think this would make me bang my head against the wall in frustration, but it doesn’t.  That’s because even if I ignore the many non-MemShrink-related benefits of CPG, there are two big MemShrink-related ones.

First, CPG will enable per-tab memory reporting, something that users have been requesting for years.

Second, CPG will lead to much more detail in about:memory and about:compartments.  For example, Nils Maier has written a patch that makes it obvious all the JavaScript modules that have been loaded.  Another example:  Justin Dolske found that plusone.google.com was doing something silly (constantly creating new iframes?) that caused huge numbers of compartments to be created;  without CPG I think all those globals would have been lumped into a single compartment and the problem would have been much less obvious.  More information in about:memory will lead to more diagnosis of existing problems — particularly leaks of various kinds — in both Firefox and websites.

Memory Reporting

Kevin Locke tweaked the JS memory reporters so that more compartments are distinguished, instead of being lumped together.  This was his first Mozilla patch — well done, Kevin!

Nathan Froyd improved the coverage of the layout memory reporters.  This significantly reduces “heap-unclassified” for huge pages like the single-page version of the HTML5 spec.

Bug counts

Here are the current bug counts.

• P1: 22 (-2/+3)
• P2: 83 (-6/+4)
• P3: 105 (-5/+6)
• Unprioritized: 3 (-1/+3)

Mostly bouncing around at the moment.

by Nicholas Nethercote at May 16, 2012 06:26 AM

Project News

• We released Add-on SDK 1.7 today. Please see the blog post and release notes for more info.
• Nicholas Nethercote’s latest blog post mentions some progress on Nightly that will mitigate the problems users could see when using add-ons packed with older versions of the SDK and Firefox 15. Although we’ve seen good progress here, there are still some underlying issues, and we strongly recommend that you re-pack your older add-ons with SDK 1.7.
• SDK developer Matteo Feretti ( aka ZER0 ) is speaking at JSDay in Verona tomorrow – let’s all wish him luck!

Quick Stats

• Total open bugs: 291
• Bugs created last week: 12
• Bugs fixed last week: 5
• Total SDK-based Addons on AMO: 745
• Open pull requests on Github: 36

Note: the stats above are based on the queries I linked to for each item. If you have suggestions on how these queries might be made more accurate,please comment below. Stats generated at 2012-05-15 08:12:38 PDT

Meeting Brief

• no Builder update today
• SDK: releasing 1.7 today, khuey’s latest patch reduces impact of platform changes on older SDK versions.
• Roundtable: landing changes to self.dat as per Irakli’s ‘Spring Cleaning’ plan.

Full minutes are available here:
https://wiki.mozilla.org/Jetpack/Weekly_Meeting/2012-5-15#Minutes

by Jeff Griffiths at May 16, 2012 03:55 AM

 hg clone http://hg.mozilla.org/dom-inspector
 cd ./dom-inspector
 hg update DOMI_2_0_12

by Jesper Kristensen at May 16, 2012 03:00 AM

by Jesper Kristensen at May 16, 2012 03:00 AM

by Jesper Kristensen at May 16, 2012 03:00 AM

Nick Cameron (feed) – Nick Cameron joined Mozilla in January 2012 and works on graphics and layout from Auckland, NZ. Previously, he has been working on research in type theory and language design.

by raccettura at May 16, 2012 12:09 AM

The Jetpack team is happy to announce the release of Add-on SDK version 1.7! You can download the SDK directly in either zip or tarball format. This version of the SDK is mostly a bug-fix release but with a few added features.

Some of the important features included in this release are:

• Plural forms for localization – many languages treat plural forms of words in different manners. The SDK’s l10n functionality now supports those forms across languages.
• Implementation of ‘Promise Abstractions’ – Promises provide an interface for interacting with an object that represents the result of an action that is performed asynchronously, and may or may not be finished at any given point in time. In the Add-on SDK, we follow the CommonJS Promise/A Implementation.
• contentStyle/contentStyleField – this give you the ability to inject a CSS file into a web page.

For more new features and a list of the bug fixes that are shipping in this release, please see the Release Notes.

As of this writing the documentation for SDK 1.7 is not available in the usual place, however they should be available by Thursday, May 17th. I will update this post when they are live, but in the meantime you have two options:

1. view them on Github: [ Tutorials & Guides ] [ Addon-kit API Docs ]
2. download the SDK, activate the SDK environment and then run cfx docs to generate your own local copy of the documentation.

As always, we’d love to hear from you about your experiences with this release. You can contact us in a variety of ways:

post to our discussion group
chat with us on irc.mozilla.org #jetpack
report a bug
check out the source and contribute bug fixes, enhancements, or documentation

For more information on the Jetpack Project check out our wiki.

by Dave Mason at May 15, 2012 10:57 PM

Like most desktop applications, Firefox is driven by an event loop. Currently this event loop is servicing a lot of events including page layout, drawing, image decoding and don’t forget JS. We do our best to handle events quickly (in a few milliseconds) and break up the ones that we know will take longer (such as image decoding). Any event, such as poorly written JS, that takes too long to process will cause the application to feel sluggish and will cause updates, animations and videos to pause.

Every web page is broken into a set of layers (backgrounds, canvas, video, web contents, position fixed element, elements that are being animated) by our layers system. When these layers are updated they must be flattened to the screen to show a final frame. This process is called ‘composition’. Currently this composition is driven by the main event loop. While compositing does add more load to the event loop from my experience the load it adds is often negligible. After all with hardware accelerated layers backends the work is mostly just queuing a few simple graphics command to the GPU.

Simplified painting pipeline (Desktop Current)

So why move compositing to a second thread if it’s relatively cheap? Because we need the event loop to be responsive to service composite events and any slow events (>15ms), such as a long JS script, will cause us to drop frames. By compositing on a separate thread we can still service layers update and keep the browser partially responsive even if the event loop is momentarily blocked by a long running script.

Simplified OMTC painting pipeline (Fennec Beta)

There is one catch however. If the main event loop is being hogged by an event then that will block certain updates, like page reflows, from reaching the compositor. However other types of updates can still be made while the main event loop is bogged down such as video which we decode in yet another thread, css/gif animations and plugins to name a few. Another use case is to support smooth panning and zooming on mobile. The goal we’re aiming for is to make Firefox more responsive.

The feature just shipped into the latest Fennec beta and provides a smoother experience while consuming less resources.

Simplified OMTC painting pipeline (Desktop/Fennec Future)

In the future we plan on landing more changes that will leverage this new architecture. We are considering many proposal shown above. Some are uncertain at this point such as having canvas workers. Right now our current focus is passing decoded video frames directly to the compositor and handling simple CSS animation asynchronously. These features will allow video and animations to perform smoothly over short periods of blocking similar to async scrolling on Fennec. We’re hoping to have some of these features landed in Q2 for mobile and desktop where OMTC will likely still be behind a preference until later this year. Stay tuned to these bugs to try out these features when they land.

by benoitgirard at May 15, 2012 10:01 PM

Head over to Fast Company to read my article on “5 Lessons For Using Open Innovation To Maximize The Wisdom Of The Crowd” - it’s one of my better ones (I think). :)

And please know - I wouldn’t have been able to write this article if not for the superior work of some of the smartest people I know. I couldn’t give them credit in my article, so I do it here:

• John Lilly, who essentially developed the framework I lay out in the article. Watch his amazing talk about the “7 Lessons from Mozilla” at Wordcamp 2009, read his blog and follow him on Twitter. He’s one of the smartest and nicest people I know.

• Asa Dotzler, who taught me Mozilla. His intuitive understanding of community and what it means to be a Mozillian formed my thinking about the power of being an open organization. You should read his blog and follow him on Twitter - he’s smart and outspoken.

• Mitchell Baker, who keeps pushing me to think about what’s important, why it matters and how we can make it better - in the open. Every time I become comfortable with where I am and what I think I know, I sit down with her. She continues to challenge me in the very best possible way. Read her blog and follow her on Twitter if you want to understand Mozilla on a fundamental level.

• Chris Beard and Todd Simpson, who in their capacity as my bosses allowed and encouraged me to experiment, try things out, risk failure and celebrate success.

• Karim Lakhani, Eric von Hippel and Kevin Boudreau, who are way out there when it comes to think about open innovation and from whom I have learned so much. I wish I could spent more time in their proximity.

• The whole Mozilla community which taught me The Mozilla Way.

I stand on the shoulders of giants.

May 15, 2012 09:49 PM

about:mozilla is a weekly round-up of news and contribution opportunities. Here’s what’s happening this week.

Firefox support questions: On the Way to 100%
If you ever need help with Firefox, the Firefox Support Forum is the place to go. In the community-powered support forum, everyone can help, including you. And everybody should be helped too. That’s why last year, the SUMO team set a goal for this year to achieve a 100% response rate, which means that every user who asked a question in our forum should get a response. But how close are we to the 100%? Find it out by reading David Tenser’s blog post or help by answering questions on the forums.

Visual Reboot of Firefox Mobile for Android

Firefox has always been a different browser, but with a recent strong wave of competitive browsers, all with similar features; design and product identity has become the differentiator. That’s why the Mozilla User Experience team revamped the design of the Firefox Mobile application for Android, and the results are stunning. Every single detail has carefully been elaborated, that’s why the new design isn’t just fluid, organic and unique to Firefox, but it also speaks strongly to the Firefox brand. Check out Patryk Adamczyk’s blog post to see what it looks like.

Windows on ARM Users Need Browser Choice Too
For the past eight years, Windows offered users a choice of browsers to navigate their digital lives. Prior to the launch of Firefox in 2004, Internet Explorer was really the only browser for Windows. Unfortunately, the upcoming release of Windows for the ARM processor could bring the digital dark ages back where users and developers didn’t have browser choices. But why does this matter to users? Check out Harvey Anderson’s blog post to find the answer.

The Web Will Connect our Future
Gary Kovacs, CEO of Mozilla, had the honor of delivering the opening keynote at a big event with a presentation titled “The Web Will Connect our Future.”. He shared with the audience that not only has the Web changed our lives once, but in mobile, the open Web is about to do it again. Check out his blog post to find out why he thinks that we all need to work together to stress the Web as a platform and to push over a few remaining hurdles like graphics and video and native device API access.

Desktop Apps with HTML5
One of the best things about HTML is that it’s never “done”. It’s been with us longer than most of the development technologies that we consider commonplace. Recently the Mozilla Apps Native Install Experience was introduced to the Firefox Nightly Channel. This functionality lets you install an HTML5 application with a native launching experience on your computer. All you need to do is to go to the Market Store, download an application and it will automatically show up in your Launchpad or in your program list. If you’re a developer, check out Joe Stagner’s blog post to see how it works.

Meet Some Mozillians
Bonjour Mozilla says bonjour to Poland, Ludovic & Zola and Janet Swisher. Read more about how these people are contributing to Mozilla.

Upcoming events
* May 16, Hotel San Marco, Via longena 42, Verona, Italy jsDay 2012
* May 16, London, England Geek Bowling
* May 19, Taipei, Taiwan JSDC.tw 2012
* May 20, Chicago (Rosemont), IL Society for Technical Communication Summit
* May 23, Melbourne, Australia Web Directions Code
* See more on the Mozilla Community Calendar

Get Involved
These are just some of the available contribution opportunities. Learn more about other ways to get involved and find other Mozillians in our community who share your interests.

About about:mozilla
The newsletter is written by Mozilla’s contributor engagement team and is published every Tuesday. For more on what has been happening this week also checkout the Mozilla Project Meeting. If you have anything you would like to include in our next issue,
please contact: about-mozilla[at]mozilla[dot]com or send us a status message on mozilla.status.net or a tweet @aboutmozilla. You can also subscribe to the email version.

Have a good week folks and keep rocking the Web!

by jan.bambach at May 15, 2012 09:16 PM

As many people know I am a fanatic when it comes to web optimization and if my blog is taking more than a second or two to load I’m freaking out because I know how important load times are to end-users and that a few milliseconds could mean loss of a potential reader or new connection.

But more importantly load times also play a major role in how search engines rank you in results because in turn they consider slow loading websites to be of lesser quality to their users and rank accordingly in their algorithms.

I have been a big fan of the folks at NetDNA which offer the service MaxCDN and recently launched CloudCache two services aimed at producing top performance when it comes to serving content on your site. Some of the top sites on the internet rely on the technology that NetDNA offers to make their websites run blazing fast 365 days of the year.

I told my followers I would give something away when I reached 2,500+ people on Google+ and I recently passed this mark as such I am going to giveaway of  Five CloudCache Basic Plans for an entire year totally free courtesy of NetDNA.

a Rafflecopter giveaway

by Benjamin Kerensa at May 15, 2012 07:04 PM

Firefox for Android launched the newly rebuilt Firefox Beta app on Google Play today! The following tutorial shows you how to install the Phony add-on for user agent switching on websites where you don’t want the mobile version.

This tutorial was created with androidscreencast.jnlp so there is added jerk and gradients that aren’t in the actual product. Enjoy the great new UI and performance (and Flash support) on your Android 2.2 phone or 7″ tablet.

by mluna at May 15, 2012 06:13 PM

During the week-end, I received a pertinent email from a former Bugzilla developer who replied to an email I sent to all reviewers about the pretty low activity in the Bugzilla project during the current development cycle. He argued that one of the reasons which made him go away, and which probably took some other contributors away as well, is our high code quality standards we have in this project. The point is that we deny review if submitted patches do not follow our guidelines or are poorly written compared to what we expect in our codebase. He suggested that reviewers should accept and commit lower quality patches, and file follow-up bugs to clean up the new code. I then brought this discussion on IRC with other core developers, and we realized how hard it is to define the right level of code quality. The problems are:

• Some new features require some rewrite of some parts of the code to be easily extendable in the future and/or to reuse some existing code. These features could also be implemented without the code rewrite, which is easier to do for the contributor, but then would be harder to maintain, or would require to duplicate some existing code, which we try to avoid as much as possible. This is how Bugzilla 2.x was written, and it took us two full development cycles (Bugzilla 2.22 + 3.0) to clean up and rewrite the codebase to be able to implement new features in a reasonable way (the 2.x codebase became too ugly and complex to easily implement anything new on top of it). So the risk to accept such patches is that the codebase would become more messy again. But on the other hand, the risk is also to make contributors leave if we keep our high standards, because a rewrite is generally not something trivial nor exciting, and contributors generally do not understand why we reject a patch which does the job.
• Accepting patches which are of lower quality and filing follow-up bugs to fix the new code before the next release seems reasonable only if the community around the project is large enough to have the manpower to do the job in the expected timeframe. The current Bugzilla team is very small, maybe 4-5 core developers + some occasional contributors helping with some not too hard bugs. This isn’t large enough to expect these follow-up bugs to be fixed promptly. But we also cannot release a new version with poor code in it, some of the reasons being security implications, performance issues, or possible regressions in some use cases. As the community is small, fixing these bugs would delay the next major release a lot, which is not a desirable effect.
• Depending on how the submitted patch is written, it would take twice the time to review + rewrite the new code to match our standards compared to the time it would take for a core developer to write the patch himself. The lack of time being the main enemy, I personally wouldn’t want to spend more time on such or such feature because it’s not written as we would like it to be. The time spent to rewrite bad code means less time to work on other stuff.

So what’s the right threshold to not make new and occasional contributors go away without badly impacting our codebase? Which rules do other Mozilla and non-Mozilla projects use to solve this problem? Please share your experience with us.

by Frédéric Buclin at May 15, 2012 05:56 PM

Design is the Differentiator.

Firefox is a premium application with a strong and distinct brand. With a recent strong wave of competitive browsers, all with similar features; design and product identity has become the differentiator. When high performance and stability are expected, a smart user experience and an emotional connection to the brand is what drives users to use the product.

Firefox design is focused on a swift, direct mobile experience. Magazine like aesthetic, less emphasis on decoration with greater focus on content and the tools to ease its access. Forms are rendered with a fine balance of organic and geometric shapes to further echo the Firefox brand messaging. Controls and layout are minimal, with emphasis on function. The aesthetic design promotes interaction with clear, direct iconography, rendered with friendly and efficient forms.

The visual design has been carefully crafted to communicate Firefox’s character subtly through a tasteful aesthetic which accents the native design language. However there are areas of the Firefox aesthetic which were designed to further maintain a unique feel.

 New Start Page & Interface Design

• Colourful layout with large thumbnails (if available).
• Unified curved tab design.
• Awesome Screen data integrated into the start page.

Visual Design Differentiators.

Tab Shape

Fluid and organic, unique to Firefox.

Awesome Screen

Unique to the Firefox experience.

Orange Focus Highlight

• Speaks strongly to the Firefox brand.
• Future plan includes the transitioning of all native (blue in Ice Cream Sandwich) highlights to our Firefox orange.

Consistency and Polish.

Through iteration and testing we have refined the interaction patterns for both phone and tablet use. Resulting in a unified Firefox experience across the Firefox landscape while being conscious of user specific needs on each platform and form factor.

The visual design first seen in the refresh of Firefox Mobile on Android will evolve into the unified style spanning across all devices (phones, tablets, tablet PCs) and on many platforms. Resulting in a purpose-built UI, optimized for portrait and landscape orientations on phones and tablets; while still maintaining a quality and consistent user experience.

Visual Elements.

Layering

As an advanced web browser, Firefox gives the user various tools and options. These options are hidden behind the web content, invoked with a gesture (swipe). The rendering of these panels is darker appearing as if they are deeper in the Z axis, beneath the core content. The transitions simulate the movement of the view port to reveal more content.

Colour

On Android often instances of colour inconsistency exist between the flavours of Android. On many devices you’ll see at least 2 colours used for focus states. On Android 4.x devices you’ll see blue and orange (for legacy apps) while on skinned devices running Android 2.3.X, the user will experience the manufacturer’s brand colour and the native Gingerbread orange.

Aim is for a cohesive user experience on Firefox properties, leading with design that is unique to Firefox and doesn’t depend on manufacturer or Google Android skinning while maintaining familiar interaction elements.

A desaturated blue (silver) acts great as a field colour both for the Firefox logo and various text / media content. Use of a saturated blue over a gray allows for a friendlier experience. The hint of blue in the grays acts as a great high contrast complementary colour when used with the orange highlight.

Texture

Texture / noise has been introduced into the backgrounds for both greater aesthetic interest and reducing banding found in subtle gradients on some lower colour depth screens.

Hints at Interaction.

In a touch interface it is expected that everything can be interacted with, but good user experience should guide interaction and not promote trial and error pecking. With subtle lighting effects, touch elements will appear closer to the user on the Z axis and appear more pressable than other items. The reverse effect is applied to input fields, subtly carved into the chrome, hinting that the form can be filled; with text in this case.

Size

Size and spacing is based on physical measurements to optimize for scalability between various form factors. Due to larger real estate tablet can afford for larger touch targets and more easily accessible actions.

Bar Height: 7 mm
Input Field Height: 5 mm
Button Widths: 5 mm
Input Field Text Height: 7 pt

Bar Height: 9 mm
Input Field Height: 7 mm
Button Widths: 9 mm
Input Field Text Height: 9 pt

Forms

All shapes and icons are rendered to have inviting feel to further engage the user:

• Forms are rendered with soft edges
• Corners possess a small radius
• Objects possess realistic shading

Pressable Items

Any item that manipulates chrome should have some 3D styling, rising above the background. This can be accomplished with a drop shadow and / or bevel.

Input Fields

Any item that acts like an input field should be inset in the surface that it is set upon. The design’s aim is to invite the user to interact with the control but not feel it will work like a button since it doesn’t appear that it can be pressed down.

by Patryk Adamczyk at May 15, 2012 05:52 PM

Starting this summer, we’re aiming to help create a group of webmakers. Building on Mozilla’s Manifesto—to promote openness, innovation and opportunity on the web—we’re rationalizing a set of core skills, developing learning objectives and outcomes associated with those skills and offering opportunities to try them out. This effort aligns extremely well with the development and promotion of #5 in our mission list: “Individuals must have the ability to shape their own experiences on the Internet.”

What’s a webmaker?
What do we mean by webmaker? Someone who contributes to the web but also someone who understands the web and its inherent power. Our focus is on moving people toward doing rather than perceiving but both are required. Experimentation is where we’re headed. Guiding people toward understanding the software that constitutes the web so that they can make more informed and educated decisions about not only how they interact with the web, but how they interact with the systems that lead to the power of the web. Yes, systems as we’ve been discussing in previous posts. (Avoiding the complex discussion of Foucault’s systems of power for now, thanks.)

Code is political
Code is political. While that may seem to be a polemical statement, it’s one that serves to inform the currently omnipresent drive to teach people to code. Code is enveloped in systems of power—systems of power that will increasingly play large roles in people’s lives. Understanding that you can create as well as consume seems a fair balance. More people having a literacy is something to be desired, not shunned or disdained. (More info here: Lawrence Lessig’s Code is Law)

What do we mean by literacy?
Traditional literacy lifts people out of poverty, modifies their worldviews, opens up new vistas and provides new opportunities for further enrichment, whether they be social, political, professional, or ideological. If you want your own proof, just search with this combination of terms “literacy and poverty.” Who’s to say that digital literacy won’t accomplish similar things? In the vein of the scientific method, why not test it out?

Literacy itself is a complex term that encompasses a broad spectrum. In our case, literacy is a basic communication skill, akin to numeracy or traditional language literacy. We’re not aiming to make everyone into Joycean code experimenters pushing the boundaries of language and comprehension, nor are we aiming to move everyone toward Hemingway-esque brevity and conciseness, but if some of you decide those pathways are for you, all the better. At least you’ll be moving forward with a broader understanding of what’s possible. And you will be making the decision for yourself, not having it handed to you by some faceless mega-corporation.

Our initial take on web literacy skills is bouncing along as an ongoing experiment (sounds familiar, right?). In the same vein as iterate often, we’re out there trying things on, seeing what feels right. Working with other organizations to leverage their understandings of web literacy and expand upon our own.

What we’re interested in doing with webmaking is shining a light into a place you may not have considered looking before. Showing you that that place is not full of monsters, is not incomprehensible, but is instead simply the exact same world you’ve been experiencing all along just translated into another language. Learning to code is a deciphering of sorts—a decoding of symbols. It offers a different lens through which to view the world.

Opportunity
This new knowledge lens may significantly alter the way you perceive the world; it’s hard to say how it will affect you. Perhaps that unknown quantity is precisely why Mozilla believes learning to code is something everyone should be afforded the opportunity to learn how to do. The operative word in that sentence is opportunity.

Knock, knock, knock.

by carlacasilli at May 15, 2012 05:23 PM

Planet Mozilla viewers – you can watch this video on YouTube.

A little more than two years ago when I joined the support team, one of the first things that struck me was that most every support procedure we had involved a long list of troubleshooting steps. The idea seemed to be, let’s try to identify the exact cause of the problem and just fix that. That sounds reasonable but the practical implication of that often isn’t: Is your software up to date? If yes, let’s turn off your plugins and see what happens. Did the problem go away? No? Does the problem happen in safe mode? If no, let’s try turning half of your extensions back on. What about a new profile? Great, now just copy places.sqlite from your old profile to your new profile.

What a mess. What mere mortal has the time, skill and patience to work their way though all that? And if the thing that needs fixing isn’t easily reproducible? Forget it. It’s now become a part-time job. I suspect that for many people, it’s just easier to switch to another browser since you’ve already got one installed on your computer. Problem solved.

So the support team worked with product and engineering to create the Reset Firefox feature. The first implementation of this is a button on the Troubleshooting Information page (about:support). What is does is create a new profile and migrate your bookmarks, passwords, cookies and form data. Everything else gets set to the defaults.

I have to say, this thing is like magic. You basically get a brand new Firefox installation without the penalty of losing all your data. This is especially useful as a quick fix for the thousands of posts we see on social media where people often express vague complaints about Firefox. “Firefox is slow.” “Firefox crashes too much.” “Firefox sucks.”

The big gap in the current implementation is that, for the most part, people won’t know about this feature unless we tell them about it. Future plans involve making it discoverable. Soon we’ll give users the option to reset Firefox when it crashes on startup for the third time. And the really big thing will be giving Windows users this option when re-installing Firefox. Maybe one day the phrase, “I tried re-installing Firefox but it didn’t do anything” will go away.

Read more about Reset Firefox on the support site and then download Firefox Beta and try it out.

Notes:
• This may be broken in Nightly or Aurora. Only try this in Beta.
• It only saves bookmarks, passwords, cookies and form data. You will lose your add-ons, Sync settings, open tabs and tab groups.
• It only works with the default profile. If you’ve opened Firefox via the command line or shortcut with a profile that isn’t the default, you won’t see the Reset Firefox button.

by Verdi at May 15, 2012 04:01 PM

Here's a short summary of Mozilla-related work I've done in week 19/2012 (May 7 - 13, 2012):

• CSI:Mozilla / CrashKill:
  More discussions and forming of a concrete plan for Rapid Betas on Socorro.
  Found an interesting signature escaping problem in Socorro.
  Did some triage of mobile crashes with Noaki and Sheila, with an eye on the upcoming beta.
  Discussed and made a proposal for Java signature improvements.
  Followed up on WebRT and B2G crash reporting process, will probably need some more poking until things actually happen there.
  I tried to move things forward on potentially blocking old Flash versions with actively exploited security vulnerabilities, and involved the security-group in the discussions. I hope we'll come to a conclusion on this now.
  Just like every week, watched new/rising crashes, caring that bugs are filed where needed.
• SeaMonkey:
  When people saw a strange website reversal to old days, I poked IT people to get this resolved and they found a temporary solution. There's a larger problem behind this which doesn't only affect our site and which they are working on.
• German L10n:
  I synched up core and suite L10n to trunk once again.
• Themes:
  While the 2.9 versions of EarlyBlue and LCARStrek got their AMO reviews, I started working on 2.10 versions fitting with current beta builds.
• Various Discussions/Topics:
  More datacenter move fallout, Android XUL Fennec display problems, stability work week planning, etc.

by KaiRo at May 15, 2012 02:34 PM

At Mozilla, our build system has a firm rule we only grudgingly violate: explicit is better than implicit. What that means is that if we depend on a library foo and we don't find it on the machine we're building on, we fail instead of silently assuming the user doesn't want to build in support for foo. If the user really wants that, she would need to pass in a --disable-foo configure flag saying so. This means we know exactly what we're shipping as binaries, and users know exactly what to expect.

Once you spend a lot of time working with Mozilla code, you sometimes forget other projects don't follow such obviously important rules. Case in point: Ruby. A default Ubuntu install builds Ruby out of the box. Of course, when you then try to do anything remotely useful:

% gem install heroku

Ruby fails with a cryptic no such file to load -- zlib (LoadError).

Turns out Ubuntu doesn't come with the zlibg1 dev library, which means the Ruby build system assumes you don't care about zlib support and happily builds without it.

Great, so you installed the library and built Ruby again, and gem actually worked. Now, you try to log in to Heroku:

% heroku login

... and Ruby fails with yet another no such file to load -- net/https error. At least the error message is slightly less cryptic this time, since it tells you to apt-get install libopenssl-ruby. Which means you need to install the library and rebuild Ruby a third time.

God knows how many more libraries the build system's assumed I don't care about.

May 15, 2012 12:30 PM

WHOA! I know, I know, I’ve totally been slacking on my “Blog Every Single Tuesday” rule. I’ve been busy. Really busy, and I haven’t had a chance to actually think about what I wanted to write. Thus, I’m doing a roundup post. Here’s some brief insight into what I’ve been thinking about and doing:

1. Linking Past and Present
   I’ve been reading a book about German Reform Pedagogy (really brief, dirty definition): the thought movement that happened circa 1900 to 1930s that changed the face of education. A LOT of the ideas we believe in for education, like learning through making, collaboration, interested-based learning, etc were being talked about in this time. A parallel thought movement (does the name John Dewey say anything to you?) called “Progressive Education” was happening in the US. They’re NOT considered the same movement by the German educational academics (even though they happened at the same time, and they were talking about the same things and concluding with the same conclusions. Also, if you click the English button on the Wikipedia article Reformpädagogik, you get the Wikipedia article on Progressive Education…) Anyway, I’m thinking about the connection between the discussion happening NOW and these movements from the 1900s. Specifically, I’m wondering if Georg Kerschensteiner, one of the leaders of the German Reform Pedagogical Movement, actually conceived the pillars of digital and/or new literacies in a time when RADIO was new fangled and cool.
2. Cultural Clashing
   I live in Europe, so I’ve been looking at the landscape of web literacy and learning programs here on the home front. I talk to people a lot about the work we do at Mozilla, and I keep wondering how we can get the awesome energy of the North American continent’s movement in the educational realm to percolate here in Europe. There’s tons of good programs happening, but it seems like the cohesion of our community here is faulty at best. There are pockets of innovation happening, but we’re not yet playing the role we’re championing in the US and Canada. In London and Berlin we’re finding ways to bring web literacy to a number of different groups, but what about the rest of Europe?When I talk to people about web literacy here in Germany, they’re sometimes skeptical, something I find pretty unbelievable. There seems to be a lot of people that still believe that web literacy skills will be gleaned without guidance, that these skills are somehow given, not learned. This misguided idea that the new generation are “digital natives” seems to be influencing the learning landscape, and I want to step up and change that notion (with a little help from my friends, naturally).
3. Guiding the Guide
   Been doing tons of thinking and work on helping informal instructors access Mozilla content. Lots of people are wanting to run their own hack jams and teach this or that techie thing to youth and adults. A lot of these people need a little help, so we’ve been creating materials that give them step by step guidance and activities that will help them hit the ground running. Call it curriculum, call it learning materials, call it hacktivity kit 2.0, the point is we’re trying to make some resources that help those active community members run their own events and teach people things without having to go crazy figuring out the all important “what will we do!?” question. We currently have a wiki up. It’s just temporary, not all slick and beautiful, but we needed a holding place for some of this stuff. You are welcome to edit and add to it!
4. Introduction to Web Native Film and StoryCamp
   StoryCamp planning, preparation and production is coming along. We’re all over it like white on rice, and we’re building some kick-ass stuff. What’s really cool is that with this theme, I get to spend a bit of time looking at crazy films from the 50s, and that is a load of fun. I also get to repurpose a robot I drew a few weeks ago.

WebPageMaker Open Web Nice Robot concept drawing:

StoryCamp Robots Invade Everywhere Evil Robot concept drawing:

Muwwwhahahha! Actually, this whole project is awesome and awesome fun, having a blast! Here, check out some of our mockups And don’t forget feedback is always welcome!

by Laura Hilliger at May 15, 2012 11:04 AM

Sparrow For iPhone Gets New Features, Push Coming Later With Yearly Subscription

http://www.cultofmac.com

Want push support in Sparrow for iPhone? You’re going to have to pay extra for it.Sparrow for iPhone — the third-party email client that has left Apple’s built-in Mail app [...]

Read more »

Credits: PUSH by Steve Snodgrass

by Arnaud Le Ruyet at May 15, 2012 10:21 AM

This is the first of a series of articles dedicated to the useful libraries that all web developers should have in their toolbox. My intent is to show you what those libraries can do and help you to use them at their best. This first article is dedicated to the Raphaël library.

Introduction

Raphaël is a library originally written by Dmitry Baranovskiy and is now part of Sencha Labs.

The goal of this library is to simplify work with vector graphics on the Web. Raphaël relies on the SVG W3C Recommendation (which is well supported in all modern browsers) and falls back to the Micrsoft VML language in order to address legacy versions of Internet Explorer. It also tries to harmonize some working issues across SVG implementations such as the SVG Animations. As a consequence, Raphaël is a very nice wrapper to produce consistent kick-ass graphics all over the Web.

Basic usage

The library has very good documentation with many examples. Do not hesitate to use it extensively.

The following example will draw a simple red circle inside an HTML element with the id “myPaper”.

// the following example creates a drawing zone
// that is 100px wide by 100px high.
// This drawing zone is created at the top left corner
// of the #myPaper element (or its top right corner in
// dir="rtl" elements)
var paper = Raphael("myPaper", 100, 100); 
 
// The circle will have a radius of 40
// and its center will be at coordinate 50,50
var c = paper.circle(50, 50, 40); 
 
// The circle will be filled with red
// Note that the name of each element property
// follow the SVG recommendation
c.attr({
    fill: "#900"
});