Customizing the Web

I use the web quite a bit, and one of the things I do quite often is customize my experience on sites I frequent. Most of that customization comes via Stylish and Scriptish (the latter a fork of Grease Monkey).

Some of the things I customize are minor, like page aesthetics. Some I consider more important. Take Google Search for example. Did you know some of your search results are censored? This is thanks to laws like the Digital Millennium Copyright Act (DMCA). Google is good enough to notify you when censorship has occurred.

But only if you scroll to the bottom of the search page. Which you may not do for quick, one-off searches. This is the equivalent to newspapers printing important news on page 8N which is otherwise a full page of ads.

But thanks to Stylish:

@namespace url(https://www.w3.org/1999/xhtml);
@-moz-document domain("google.com") {
  #mfr {
    position: absolute !important;
    top: -20px !important;
    right: 0px !important;
    width: 300px !important;
    background-color: #FCC !important;
    font-size: 1.2em !important;
  }
}

This adds a bit of styling to the notice and throws it at the top-left of the search results.

With most software this wouldn’t be possible. Without the open web that notice (if there at all) would be permanently stuck down at the bottom. Maybe I would adopt the practice of always looking for it, but it’s doubtful.

Instead, I can remain aware of the prevalence of censored results. I can find out more.

The challenge is opening up this technology to those who don’t know CSS, JavaScript, HTML, or how to debug those technologies (as modifying the page often requires understanding it rather than simple repositioning like above). There are ways to do that, but so far they seem limited to having programmers hand ready-made solutions to novices (eg, via sharing snippets like that above).

The problem with that approach is that it limits the idea pool to those who either can implement a modification themselves or knows someone who can.

It Came…From the Internet! MutationObserver

[Fade in Louis Armstrong singing On the Sunny Side of the Street.]

Grab your lab coat // Put on your hazmat, baby
Leave your worries on the doorstep
Life can be so sweet
On the sunny side of the street
[…]

Okay, MutationObserver isn’t really that scary. It’s part of the DOM4 specification as a replacement for Mutation Events. MDN: DOM: MutationObserver documents it.

What are Mutation Events? A DOM3 specification to allow you to watch the DOM change using the well worn events API. You know, the old

node.addEventListener('click', onNodeClick, false);

style events.

But Mutation Events made things slow. A lot of sites mutate or change the DOM quite often, and depending where you attach the listener and how frequent the changes come, it added up to a lot of event traffic. MutationObserver replaces this.

A MutationObserver doesn’t receive a callback for every change. Instead it receives periodic callbacks for a group of changes. That group may contain a single change, depending on the stability the DOM tree (if only a few changes occur at large intervals, they may be announced separately). Think of it like an announced log instead of alerts.

MutationObserver takes a bit more setup than just tagging an event listener on a node. But that setup makes it cleaner and more useful.

var observer = new MutationObserver(function (mutations) {
    mutations.forEach(function (mutation) {
        console.log(mutation.type);
    });
});

That’s a simple MutationObserver. You provide a function that receives an Array of MutationRecord objects. But that just creates the observer, you have to activate it.

var config = { childList: true, attributes: false, characterData: false, subtree: true };

So first you need an object (technically a MutationObserverInit object) specifying what types of mutations you care about (you can also specify some extra options about which data to include in the MutationRecord objects you get back). You really just worry about making this give you what you want and not give you anything else. If you want to know about node added or removed from the DOM tree, you don’t need attributes or characterData, for example.

Once you have your configuration ready, you also need a root node to observe. You can observe document, the true root, but you should attempt to observe the minimal subtree you will need.

Say the page shows a chess game, and you want to add some automatic comments or classes (for styling) to the move log. You only need to observe the part of the tree that contains the log. The page logo, the board itself, the chat, and anything else don’t matter.

var moveLog = document.querySelector('#moveLog');

We can finally turn the MutationObserver on:

observer.observe(moveLog, config);

Now when new changes happen under moveLog, the function we specified when creating the observer will be called back. But we can go a bit further. Let’s say you have two separate subtrees to observe. You could either find their lowest common ancestor and observe that whole subtree, or you can do the smart thing and simply register the observer with the second subtree:

observer.observe(oneNode, config);
observer.observe(twoNode, config);

Note that was, “register the observer with” the node, not the other way around. Although you call on the observer, this tells it to add itself to the list of observers associated with the node in question.

You can also shut the whole thing down:

observer.disconnect();

This kills all the registrations, so unfortunately you cannot easily modify the actively observed subtrees.

Also note:

observer.observe(myTarget, config1);
observer.observe(myTarget, config2);

This replaces the options for observing myTarget.

Anyway, that’s a brief overview of the not-so-scary, highly useful MutationObserver. It’s particularly nice for user scripts that want to cope with dynamic web pages. Instead of being stuck with some modification only for the nodes that were there when the page loaded, you can now easily observe changes and carry your modifications over as new nodes appear.

Learning to Compute

Some well-known person in the software community wrote about why, I don’t know, mere mortals?, shouldn’t learn to write software.

To be honest I didn’t read it. Any time someone says that I shouldn’t do something for any non-scientific or non-moral reason, I add another check to the column for “this maybe should be done.”

I skimmed several of the thoughtful rebuttals on other blogs, and think their general consensus is valuable, which is basically, “yes, you should learn to write software, because it’s awesome.”

Anyway, this isn’t about whether you should or shouldn’t, can or can’t, will or won’t. This is about doing the actual bit that’s being discussed abstractly.

Let’s assume for the sake of argument that you are using Firefox and viewing this site in the browser. Let’s assume it’s on a non-mobile Operating System with a keyboard and other fancy stuff like a screen.

Let’s assume you were to press the key combination Ctrl+Shift+K (Hey! Wait a damn minute there. Don’t press stuff without finding out what it does!) or go to the Firefox menu, to Web Developer and then to Web Console (which is what the key command before does).

You should now see some slick little bit of chrome fall out of the sky above the website.

You should see that it has a text input area at the bottom. Click there. Type something like, alert("Boo!"); and press Enter. Stop! Why are you typing stuff in before you know what it does?

Okay, to be fair, I type stuff in without knowing exactly what it will do, and aside from some wizards, most people who write software do this. But we know almost exactly, and we’ve got experience: think of it like cooking. A master chef can imagine how a recipe will taste just by reading it. Experience works like that, though even the experienced can be mistaken.

So before you type something, you should stop and think about what it might do, and what you want it to do. alert, that might make lights flash and buzzers sound. But it has those parentheses around the quoted bit.

Maybe it creates some kind of alert using that quoted part.

Go ahead and try it.

Right! A happy little alert box pops up with the text Boo!.

So why does that work? Think of it like cooking, again. The baker doesn’t just reach into the oven and pull out a pie. Each piece had to be made, including the pie tin (which was beyond the scope of the baker’s activity). In this case, think of the alert() bit as the pie tin. The makers of the browser gave you the pie tin and said, “fill it with what you need to fill it.”

If you think about it, the parentheses almost look like a pie tin. Are you hungry yet?

So let’s do something a bit simpler this time. Just type 1 + 1 and press Enter. Right, good thought, we have to ask what it might do. Well, if I wrote it down on paper and handed it to you, what would you probably say? Yes, you would say, “two.” That’s what the console will say back to you:

[13:25:34.419] 1 + 1
[13:25:34.423] 2

That bit there is taken directly from the console. If you select lines in it and then copy them, it puts the fancy timestamps in front. Those help you understand the behavior of more complex programs by letting you know what times the output corresponded to, how long things took (by seeing the difference in the times), etc.

Okay, one more thing to try for today. Start typing document, but just the d at first. Notice it gives you a handy drop-down of possible matches. You can use the up and down cursor keys to go through that list, or click on one with the mouse. If you keep typing, once it has a plausible match it will ghost-in the rest of the word for you. Once that happens, you can press the Tab button to complete the word.

Once you have document, put a . after it, and then type getElementById. You may have noticed that there were other getElement* bits, and some other things too. I hope you noticed, that’s the main skill to work on as a programmer: seeing all the details and wondering why.

Later on, if you want to keep learning about programming, you might go back to the web console (Ctrl+Shift+K) and just start typing things (don’t press Enter, just type say, c and see what pops up in the list). Ask yourself what they might do, and then use a search engine to ask the web what they actually do.

Okay, back on track: document.getElementById(???) What do we need to replace the ??? with? You’ll see the so-called “CamelCase” names in software a bit, so get used to mentally putting spaces in: get Element By Id. Right, if we hand it the Id, it will hand us the Element.

Let’s use one I know: site-title. That’s the id for the HTML element containing the blog title up top. So let’s do something with that.

We can do this two different ways. One way is to store the element we get back, and then speak about that, the other way is to do it directly.

The first way is to give the Element a name. To do this we can type var titleElement = document.getElementById('site-title'); and press Enter. We type var first, because we want it to be a regular variable. This has to do with scoping rules, which I won’t discuss here, but you will learn about if you keep growing your skill.

Once we have it as a variable named titleElement (you could name it something else if you prefer), we can use the name to reference that Element.

Now we can type titleElement.style.backgroundColor = "black"; and press Enter. The style part means that you want to access the Element‘s style properties. The style.backgroundColor means you want to access the background color of the Element. And we’re setting it to the named color black.

Scroll up to the top and look at the result. The background around the blog title is now black!

It doesn’t look right, of course. The background was more of a white, so let’s try changing it back.

Press the up arrow, and it will show you the last thing you typed. You can navigate the input history in the console using the up and down keys.

Change black to white and press enter again. Oops, it’s not quite white, either. What do we type to get it back the way it was?!

Let’s set it back the way it was, but let’s do it the other way I mentioned before: type document.getElementById('site-title').style.backgroundColor = "inherit"; and press Enter. Here we aren’t bothering to store the Element in a variable. We’re using it directly from the return of getElementById(). And we’re setting the color to inherit, which means “whatever the parent value is.”

The rules for Cascading Style Sheets are something you can learn about later, but you can think of it like cooking again. By default, a BLT sandwich has bacon, lettuce, and tomato. If someone wants a special one, then maybe its bltSandwich.style.bread = "pita", but normally it’s just whole-wheat. inherit just means, “whatever it would be for the environment it’s in.” (It means inherit the value from the parent, because elements in HTML are in a tree structure).

The main thing is that the color is back to normal!

That’s enough for now. You can close the console by clicking the x in the corner of it, or by pressing Ctrl+Shift+K again. If you make a live change while learning (live changes are those that you make to a webpage directly, such as by the Web Console), and you don’t know how to undo it, simply reloading the page should get you back to the default state.

Slow JavaScript in My Browser?

Another short one for November. Firefox 9 beta is now out, and that means the awesome package manager for Iceweasel has already got it packaged for Debian Unstable. In reading up on what’s new in Firefox 9, people mentioned Type Inference being added to its JavaScript engine, bringing some nice performance gains.

That led me to want to see how fast my browser is these days. For that I hopped on to the Kraken benchmark and V8 benchmark (separately, of course, to prevent one from slowing the other). Then I became concerned.

My everyday browser and profile:

  • Kraken: 12008ms
  • V8: 1522

When I tried Kraken from Firefox Safe Mode, it actually got so bad that the imaging: gaussian-blur portion of the test was timing out. For the sake of consistency/thoroughness, I’ve repeated the benchmarks from Safe Mode:

  • Kraken: 59363ms
  • V8: 583

(Note that on the Kraken tests where it caused the ‘slow script’ dialog to pop up, I had to manually tell it not to issue the warning the first time, which means a score even worse than it would have been (humans are slow).)

[Edit: Mozilla Bugzilla: Bug 453642 shows that Just In Time (JIT) compilation of JavaScript is disabled in Firefox Safe Mode, which clears that up.]

As my local builds of Firefox are on version 11 (the current nightly), using one of them won’t give me the exact same code to test against, but it’s a start. I can also download a copy of the Firefox 9 beta as compiled by Mozilla, and run that, to give me another datapoint.

Why am I doing this? Because at this point, it’s not obviously a bug. It could be caused by some other problem with my system, and even if it is a bug in Iceweasel, the extra data can prove useful.

So:

  • Iceweasel with my original profile:

    • Kraken: 12008ms
    • V8: 1522
  • My local build:

    • Kraken: 3612ms
    • V8: 5476
  • Official Firefox 9 beta:

    • Kraken: 3439ms
    • V8: 6324

So we know that there’s a problem. Now the question is whether it’s with the package itself or something else on my system.

Here I try Iceweasel with a fresh profile:

  • Kraken: 3711ms
  • V8: 5696

Okay. So now I know, no doubt, that it’s my profile. What’s disturbing is that the problem still showed itself in safe mode.

My first step in tracking down my profile’s problem will be to disable all add-ons:

  • Kraken: 3693ms
  • V8: 5691

Ouch. Still not sure why safe-mode would have degenerate behavior, but at least I can now do a binary search (disabling half the add-ons, and test, etc. until I narrow it to a single add-on) to find which add-on is causing my problem.

And the winner is… Firebug. Firebug is an excellent extensions that lets the user inspect the web they use every day, but to do that it has to have its hands in the pie of the web, which can slow things down. Let’s see what we can tweak about my current settings for Firebug to hopefully leave it enabled while getting acceptable performance.

And a search turns up Mozillazine Forums: TM and PM Performance Thread: Comment by phuzi0n:

Is Firebug killing javascript performance after TI landed for anyone else? I tried updating to the latest stable Firebug 1.8.2 and then to 1.9.0a1 but it still slows down v8 benchmark by ~4x and sunspider by ~2x. I made sure that Firebug’s javascript profiling is disabled (with Firebug profiling enabled v8 is ~10x slower).

So, the current vintage of Firebug has some issues with the changes to JavaScript. That’s unfortunate, but knowing is half the battle.

In retrospect, I should have tested with a fresh profile before worrying about other builds of Firefox, as it would have highlighted the source of the problem much faster, but I was thrown off by the really bad performance in safe-mode, which led me to believe it couldn’t be an add-on.

For the time being, I am going to keep Firebug on, but if I run into pages that seem slow, I can always turn it off.

The World-wide Web

The World-wide Web, or the WW as it is known.  If but for a hyphen all those years ago, how many less characters would be have to type to navigate this landscape?!

Alas, we still type www; or some of us do and some use various shortcuts.  Some of us are on mobile devices that predict we’re about to text someone about a website and promptly display out-of-area…

To the point: we’re all mostly just using the web, and we’re all using it in more diverse ways than ever.  So, what is the important thing?  There are several.

Screens

A couple weeks ago the NYT Magazine was devoted to screens.  More than ever our monitors and TVs have grown while our cell phones and laptops have first shrank and then grown and shrank and grown.

The important thing here is that there’s less telling than ever what size screen a web page will be viewed on.  The future is more of the same in that regard, but as small devices get larger screens and as resolutions improve this becomes less important.

Revolution

The advent of AJAX and the long-entrenchment of Adobe Flash have made the web crunchier.  Parts are still the old web, and the standards there have rapidly improved.  The AJAXian incursion broke that: some sites have it, some don’t.  Some sites implement it well, some don’t.  Above all, it presents a stumbling block for standardized interaction.

Example: in gMail I can click a checkbox, hold shift, and click another checkbox with the result being the intervening boxes become checked.  So, then, in site example.com can I expect this?  I can expect all I want, but the actual behavior may not be consistent with my expectations.

The Browser Congress

In many ways the overarching state of web browsers has improved (for those who install or upgrade their browsers).  There is less of a browser war and more of a congress these days.  The main (and longstanding champion) crumudgeon is Microsoft with Internet Explorer.  Don’t expect that to change, except in that their marketshare is slowly but surely declining.

The important thing here is that the overall state of the web is in flux: it’s improving on many levels, but there are new concerns.  Rendering engines are rapidly approaching perfection (curmudgeons aside), so I won’t discuss them.

The Javascript Push

The new push inside the browser is for more, better, faster Javascript engines.  The goal is to provide outstanding performance for AJAX and other Javascript uses.  The general path is to provide JIT compilation to native code for execution.  You may see stories with titles like “Firefox kicks Chrome in the knee, but Webkit takes a cheese grater to both of their knuckles.”

The important thing to take away from these is not that any browser is particularly better, but that they are all better.  One may be a little faster, but they are all moving toward an acceptable performance plateau.

The Plugin Push

The other big push has one foot in and one foot out of the browser: Flash, Silverlight, Java.  Well, Java hasn’t been pushed as Silverlight and Flash have been.  Flash is the current top dog, though the Java plugin is probably nearly as widespread.

The problem is that if SVG support were up to snuff these plugins wouldn’t really need to exist to the extent they do.  With the aforementioned JS engines faster than ever they should be fine for most purposes.  With Silverlight and Java there is access to some outside code paths.

Java is the winner of the three in my book: as far as I am aware it is the closest of the three to a conduit to allow execution of code without adding a bunch of specific content/functionality.

Silverlight is intermediate between Flash and Java plugin.  It still provides some of the unnecessary things that Flash does, but it also provides access to .NET language use from what I know of it.

Flash has entrenchment going for it.  It has added better non-Windows support, which is good too.  But above all, the use of a plugin should either be to provide a full application or to provide a bare connector to be built on top of.

The Soft War

The soft war is roughly the idea that ‘Internet’ is becoming (? is?) synonymous with the web.  The internet supports so much more, and while we still use some aspects, we use them less than we used to and with less awareness than we used to have.

This is good for Google, as they are a web company: they are foremost interested in indexing and advertising in your browser.  But in many ways it’s bad for the users: it places a big bet on browser security and on anti-phishing technology.

The important thing to understand about the web, for it to remain a dominant technology, is that it must:

  • continue to give the user the power
  • be augmented with user-friendly encryption technology
  • gain data portability sanity

Anyway, that’s what I’m thinking about today.