End of Bookstack, but Looking Forward to Firefox 57

Back in 2007 I was a Firefox user and wrote my extension Bookstack, which is now dying due to the changes to Firefox. But I am looking forward to the improvements Firefox brings, even though this seems like the end of an era of extensibility in the browser.

Why is Bookstack done?

My own browsing habits have changed since I wrote it. In recent years, I’ve continued to use Bookstack, but more as a speed dial than as it was originally intended as an inbox system for links. I’ve thought about writing a new sidebar to do something that suits my current usage, but for now I’ll see how life without Bookstack is before I embark on another extension.

There are some users of Bookstack out there, and I’m sorry I won’t be able to support them, but the source is available if anybody wants to take it up. The fact is that under the changes to Firefox, Bookstack would require a full rewrite anyway, and it would lose features in the process. The main painpoint would be the UI.

In the early years, Bookstack did most of its own work to build the sidebar until I worked in XUL long enough to realize I could piggyback on Firefox itself for a lot of that code, which reduced the maintenance burden on Bookstack considerably. With the change to webextensions, that’s no longer the case.

I enjoyed the project while it lasted. Ten years is a good time for it to retire.

Why Firefox will still rock

The change that Firefox is making is the first step toward a next-generation browser in terms of speed and memory use. I haven’t tested the 57 beta yet, but it’s purported to be fast. That’s great, and the changing to webextensions reduces the burdens on Firefox to let it continue to improve much more in the years to come.

End of an era

But that change comes with a cost, as mentioned with my own EOLing of Bookstack. The customizability of the browser is being limited. It’s not the Fisher-Price Apocalypse some might fret over—that won’t happen as long as the underlying browsers and protocols have open source roots—but it is limiting.

Browsers are supposed to be agents for the user. They are supposed to do the user’s bidding. Limiting the ease of modifying the agent isn’t great, but other limitations have always thwarted some types of user choice, whether it’s each browser keeping its own data (with some ability to import/export between them), or browser security getting in the way of the user (there’s an inherent clumsiness in trying to interact with iframes in userscripts, for example).

Return of the User

The next act for the web will hopefully be a resurgence in users finding new ways to work around the limitations of browsing and webextensions. There are always new threats to the dream of a web that serves users, and Google Chrome has invited a certain amount of complacency among the multitude. With a bit of luck, a resurgent Firefox will help to ignite a new generation to work for an open web again.

The Closing Web

Taking a break from discussing the FDA’s proposed deeming regulations to talk about the now-released FCC proposal for regulating ISPs and the announcement by Mozilla that they will ship EME (Encrypted Media Extensions).

EMEs in Fx

First, what will Firefox include? They will include the W3C’s EME standard for HTML5 video. This standard effectively says that an implementing browser includes a plug or a mount for DRM. The browser doesn’t have to include DRM directly (though it appears a browser vendor could ship it directly).

Think of it like a car, and because of car theft, a trade group passes a rule requiring members to include remote-controlled self-destruct mechanisms in their cars. Except they didn’t require the car makers to build-in the actual explosives. They just have to provide a place to put the explosives and the remote-detonation functionality to blow the car up if someone installs the explosives.

And then let’s say that all the fast food drive-thrus said you can’t buy our food unless you have the self-destruct system enabled. That’s you going to ACME Entertainment and streaming the movie, getting the popup that says, “please install this EME plugin.”

We’ve seen this before, with codecs. Mozilla resisted including H.264 because it’s a proprietary codec that isn’t available for all systems. But other major vendors paid for it and shipped it without blinking, and sites put videos out in H.264. Mozilla did what they felt they could, but eventually began relying on operating system support for H.264.

Mozilla is a large organization, risk averse. They do not want to see other browsers force them into a less influential position, potentially causing even more harm to the web. So they run the numbers, hold their nose, and compromise if they think it’s a bad path that may let them get to a better place to fight tomorrow. In other words, they see the risk of DRM entrenchment as less likely or less harmful than Firefox being left behind by users who increasingly watch video in a browser.

DRM serves no real purpose, and at-best represents a gris-gris for parts of the entertainment industry that do not innovate adequately. Valve Software and some other video game creators, are just starting to recognize the economic benefits of openness and artistic community. These are promising signs. As the lines blur of the lines between video games and film/television, it is expected that other industries will follow and that DRM will become rarer and rarer.

FCC’s NPRM: “Protecting and Promoting the Open Internet”

The actual proposal (FCC: PDF: 15 May 2014: Protecting and Promoting the Open Internet) only contains a few rules:

  • Transparency
  • No Blocking
  • No Commercially Unreasonable Practices

The rules that aren’t yet proposed have raised the public’s ire. The proposal requests comments on a variety of issues, taking a “we’ll make the rules later” approach. Early on in the proposal (p. 3) the FCC acknowledges two paths seem viable (sec. 706 and Title II) and they want comments on the best way forward.

Currently the FCC classifies ISPs as information services, and the court that struck down the previous rules said, obiter dictum, that they did not believe section 706 would allow for certain regulations unless the FCC reclassified ISPs. This is not a binding ruling, but should be taken as weight against merely trying to shoehorn non-common-carriers into regulations under section 706.

If you read the definitions of both “information services” and “telecommunications services” I think it’s clear which ISPs should be classified as. Despite the claim of ISPs that they will refrain from innovation if classified as common carriers, they should still be so classified.

If we need “fast lanes” they can be done through some alternate arrangement that is voluntary by the information service, rather than mandated by an ISP (similar to how you can have expedited shipping by a common carrier). Or the ISPs can negotiate for a new classification by statute that will include, e.g., mandatory progress and innovation, restrictions on operating as an ISP and line owner and media company simultaneously, etc.

Currently, the only meaningful way forward seems to be for the FCC to classify ISPs as telecommunication services subject to common carrier rules.

Customizing the Web

I use the web quite a bit, and one of the things I do quite often is customize my experience on sites I frequent. Most of that customization comes via Stylish and Scriptish (the latter a fork of Grease Monkey).

Some of the things I customize are minor, like page aesthetics. Some I consider more important. Take Google Search for example. Did you know some of your search results are censored? This is thanks to laws like the Digital Millennium Copyright Act (DMCA). Google is good enough to notify you when censorship has occurred.

But only if you scroll to the bottom of the search page. Which you may not do for quick, one-off searches. This is the equivalent to newspapers printing important news on page 8N which is otherwise a full page of ads.

But thanks to Stylish:

@namespace url(https://www.w3.org/1999/xhtml);
@-moz-document domain("google.com") {
  #mfr {
    position: absolute !important;
    top: -20px !important;
    right: 0px !important;
    width: 300px !important;
    background-color: #FCC !important;
    font-size: 1.2em !important;
  }
}

This adds a bit of styling to the notice and throws it at the top-left of the search results.

With most software this wouldn’t be possible. Without the open web that notice (if there at all) would be permanently stuck down at the bottom. Maybe I would adopt the practice of always looking for it, but it’s doubtful.

Instead, I can remain aware of the prevalence of censored results. I can find out more.

The challenge is opening up this technology to those who don’t know CSS, JavaScript, HTML, or how to debug those technologies (as modifying the page often requires understanding it rather than simple repositioning like above). There are ways to do that, but so far they seem limited to having programmers hand ready-made solutions to novices (eg, via sharing snippets like that above).

The problem with that approach is that it limits the idea pool to those who either can implement a modification themselves or knows someone who can.

Selectively Cleaning Firefox History

Firefox does a nice job of keeping my history. But not a perfect one: certain history gets in my way, and I didn’t find a good extension to automatically clean it. So I wrote a python3 script that lets me do it manually.

GitHub: Gist: clean_places.py: Python 3 script to manually remove some Firefox history items.

#!/usr/bin/python3

# The queries operate only if either:
#  * There are more than 10 visits (for things like my mail, which only bug me
#    once they are trying to clog up my new tab page)
#  * They are > 10 days old (for things like searches; recent searches aren't
#    a big problem; old searches are).

# This should be run only if Firefox is shut down (and I mean really; you
# don't want to corrupt your DB because it was hung!)

# Also, take care about your queries!

import datetime
import sqlite3
import time


PLACES_DB = "/EDITME/path/to/places.sqlite"

conn = sqlite3.connect(PLACES_DB)

cursor = conn.cursor()

select_statement = """select url from moz_places
                      where url like :query and
                      (last_visit_date < :ten_days_ago or
                       visit_count > 10)"""

delete_statement = """delete from moz_places
                      where url like :query and
                      (last_visit_date < :ten_days_ago or
                       visit_count > 10)"""

# The time stored in the DB is in UTC, so we need to be consistent.
old_time_dt = datetime.datetime.utcnow() - datetime.timedelta(days=10)
raw_time = time.mktime(old_time_dt.timetuple())
ten_days_ago = int(raw_time * 1000000)

queries = ["%localhost/roundcube/?%", # Round Cube
           "%.google.com/search?%", # Google Search
          ]

# Set to True if you want to preview what would be deleted
TEST_MODE = False

if TEST_MODE:
    statement = select_statement
else:
    statement = delete_statement

for query in queries:
    variable_bindings = {"query": query, "ten_days_ago": ten_days_ago}
    cursor.execute(statement, variable_bindings)
    if TEST_MODE:
        for row in cursor:
            print(row['url'])

if not TEST_MODE:
    print("Deleted {} rows.".format(conn.total_changes))

conn.commit()
cursor.close()

To use it you first shut down Firefox. You should make sure it’s really shut down, as you don’t want to try to modify the database while something else is. If you want to be extra paranoid, make a fresh backup your places.sqlite file.

You need to edit the PLACES_DB variable to point to your places.sqlite file. It’s usually somewhere like: /home/YOUR_USER/.mozilla/firefox/YOUR_PROFILE_FOLDER/places.sqlite.

You need to edit your query strings (queries variable). Use % as a wildcard match.

You should set TEST_MODE to True and run the script once to make sure it’s hitting your targets. The two statements are the same except one is a SELECT and the other a DELETE.

TEST_MODE will simply print what the script would delete.

If you’re satisfied, you can turn TEST_MODE off and let it rip.

If you’re adventurous you can install the sqlite3 command line tool and open the database and explore. Note that you should also ensure Firefox is not running (at least not on the same profile) when opening your places.sqlite in sqlite3.

Looking Forward to the Future of Iceweasel

Mozilla Persona

The biggest feature that I really hope takes off will be Mozilla Persona, which is will bring the ability to replace all the Login with Facebook and OpenID and remembering a million passwords with a system that allows you to have multiple managed identities with your choice of identity provider.

In many ways BrowserID is an evolution from OpenID, but as it gets built-in to the browser, it should bring an easier adoption curve with it. This can’t happen soon enough, with more and more major sites being cracked and the user data being strewn across the web.

Australis

The visual refresh of the browser (see mockups: Mozilla: shorlander: Australis Design Specs for Linux) is going to be great. My favorite part of this is the non-active tabs having low-volume to their appearance. This gives a much nicer feel and the impression that they are truly in the background; your active tab is what everything below it is about.

That will be improved over time as other features of the browser enhance that contextual choice.

GCLI — the Graphics Command Line

This is a developer tool, which allows you to easily access developer tools and commands so that you can try things out while developing on the web platform. It should allow for things like color manipulation, screenshot creation (great for documentation), and other tasks. Documentation at MDN: Tools: GCLI, which leads me to the final thing about Mozilla I’m really looking forward to:

MDN Kuma Switch

This isn’t really in the browser itself, but it’s just as important: the Mozilla Developer Network (MDN) is moving to a different platform, which will allow it to boldly go where no wiki has gone before. I use MDN a lot for learning about the browser and the web, and I’ve even made some minor edits to it before. But the current (soon-to-be-old) platform has what I consider a clunky interface for editing articles, and it’s likely it’s deterred some people from contributing.

The new MDN is based on the same codebase as the awesome Mozilla Support (SUMO), which means that further progress can be shared between both sites.

Really, some great work is afoot. I look forward to seeing what’s next.