The site uses cookies that you may not want. Continued use means acceptance. For more information see our privacy policy.

## Numerical Step Function in Python

Technical article about creating an interval/range in Python.

The classic `range()` (and in 2.* series, `xrange()`) is useful for getting an iterator of numbers. Its full signature is: `range(start, stop[, step])`.

So you can do, e.g., `range(5, 10) = [5, 6, 7, 8, 9]`.

Or you can do, e.g., `range(6, 13, 3) = [6, 9, 12]`.

But as far as I know there’s not an easy, built-in way to iterate over a range-like set of integers defined both by the range and a number of parts desired.

An example: you want five evenly-distributed numbers starting with `1` and ending with `10`. So something like `[3, 5, 6, 8, 10]`.

In my case, I had two similar use-cases. The first was the example above: a semi-arbitrary set of values in a range. I didn’t need to strictly include the endpoints in the values, but wanted a decent distribution of the values between `start` and `end`.

The second case was a little different, in that I wanted the range to always include the ends (it was a case of covering a whole range), but I also wanted to know how much each “step” over the range amounted to.

In the naïve version of this case, you don’t need the magnitude, as you could cheat and throw an extra piece in to account for slight differences (e.g., `11 / 3 => 1, 4, 7, 10` with the last piece being 10 through 11).

But there’s a nice way to evenly distribute the extra pieces: using rounding of the fractional value to distribute the extras.

Example:

``````10 / 4 = 2.5
0 * 2.5 = 0.0
1 * 2.5 = 2.5; round(2.5) = 2
2 * 2.5 = 5.0
3 * 2.5 = 7.5; round(7.5) = 8
[0, 2, 5, 8]
``````

(In Python, `round(number[, ndigits])` of `n.5` goes to the even side (when using `ndigits=0` or with a single argument).)

In this case, the caller could buffer the previous value and calculate the gap/step itself, but this is Python, so we might as well give it a mode to get that itself.

Without further ado, this is what I came up with:

``````def equal_parts(start, end, parts, include_step=False):
part_size = (end - start) / float(parts)
for i in range(parts):
part = start + round(part_size * i)
step = start + round(part_size * (i + 1)) - part
if include_step:
yield (part, step)
else:
yield chunk
``````

It’s messier than it needs to be, due to its dual-use nature. It’s arguably cleaner to have a second function that would handle the `include_step=False` case:

``````def equal_parts_only(start, end, parts):
for part, step in equal_parts(start, end, parts):
yield step + part
``````

That function would remove the conditional business at the end of the original `equal_parts`:

``````def equal_parts(start, end, parts):
part_size = (end - start) / float(parts)
for i in range(parts):
part = start + round(part_size * i)
step = start + round(part_size * (i + 1)) - part
yield (part, step)
``````

In the stepless version, it’s got another nice property: what if you do `equal_parts(0, 10, 11)`? You get: `[1, 2, 3, 4, 5, 5, 6, 7, 8, 9, 10]`. That’s a nice property: getting more parts than integers in the range.

I wrote a GIMP plugin to create stepped (or random) gaussian blurs on an image. The stepless version lets me create the set of blur levels, while the step-including version lets me properly select (mostly-)even parts of the image.

Here’s an image that used this plugin containing a dual use of this function:

If anyone wants a copy of the plugin, let me know and I’ll put it on `Github` or such.

## Selectively Cleaning Firefox History

Small script for removing some items from Firefox history.

Firefox does a nice job of keeping my history. But not a perfect one: certain history gets in my way, and I didn’t find a good extension to automatically clean it. So I wrote a `python3` script that lets me do it manually.

GitHub: Gist: `clean_places.py`: Python 3 script to manually remove some Firefox history items.

``````#!/usr/bin/python3

# The queries operate only if either:
#  * There are more than 10 visits (for things like my mail, which only bug me
#    once they are trying to clog up my new tab page)
#  * They are > 10 days old (for things like searches; recent searches aren't
#    a big problem; old searches are).

# This should be run only if Firefox is shut down (and I mean really; you
# don't want to corrupt your DB because it was hung!)

# Also, take care about your queries!

import datetime
import sqlite3
import time

PLACES_DB = "/EDITME/path/to/places.sqlite"

conn = sqlite3.connect(PLACES_DB)

cursor = conn.cursor()

select_statement = """select url from moz_places
where url like :query and
(last_visit_date < :ten_days_ago or
visit_count > 10)"""

delete_statement = """delete from moz_places
where url like :query and
(last_visit_date < :ten_days_ago or
visit_count > 10)"""

# The time stored in the DB is in UTC, so we need to be consistent.
old_time_dt = datetime.datetime.utcnow() - datetime.timedelta(days=10)
raw_time = time.mktime(old_time_dt.timetuple())
ten_days_ago = int(raw_time * 1000000)

queries = ["%localhost/roundcube/?%", # Round Cube
]

# Set to True if you want to preview what would be deleted
TEST_MODE = False

if TEST_MODE:
statement = select_statement
else:
statement = delete_statement

for query in queries:
variable_bindings = {"query": query, "ten_days_ago": ten_days_ago}
cursor.execute(statement, variable_bindings)
if TEST_MODE:
for row in cursor:
print(row['url'])

if not TEST_MODE:
print("Deleted {} rows.".format(conn.total_changes))

conn.commit()
cursor.close()
``````

To use it you first shut down Firefox. You should make sure it’s really shut down, as you don’t want to try to modify the database while something else is. If you want to be extra paranoid, make a fresh backup your `places.sqlite` file.

You need to edit the `PLACES_DB` variable to point to your `places.sqlite` file. It’s usually somewhere like: `/home/YOUR_USER/.mozilla/firefox/YOUR_PROFILE_FOLDER/places.sqlite`.

You need to edit your query strings (`queries` variable). Use `%` as a wildcard match.

You should set `TEST_MODE` to `True` and run the script once to make sure it’s hitting your targets. The two statements are the same except one is a `SELECT` and the other a `DELETE`.

`TEST_MODE` will simply print what the script would delete.

If you’re satisfied, you can turn `TEST_MODE` off and let it rip.

If you’re adventurous you can install the `sqlite3` command line tool and open the database and explore. Note that you should also ensure Firefox is not running (at least not on the same profile) when opening your `places.sqlite` in `sqlite3`.

## Future Software

A look at some ideas for better computers.

Occasionally you read interesting ideas for the future of software. Here are a few to ponder.

## docopt

docopt is basically a domain-specific language. The language is the standard `Usage` output of a command-line program. By simply documenting a program’s usage, it aims to provide the functionality of parsing the program’s invocation.

This is a great step forward in finding ways to reduce duplicated effort in an elegant way. Before, you might programmatically add or modify the argument handler, and that might generate the `Usage`. But it was still tedious, as it required keeping what was a messy bit of code as clean as you could. Messy because you had to do a lot of thinking about the basic form of your command line API, and then translate it to code.

Instead, with `docopt` you should be able to purely focus on the form of the `Usage`.

I think that this will be a common theme for future computing, without relying on more complex systems like SOAP.

## Why do we need modules at all?

Erlang.org: Erlang-Questions: 24 May 2011: Joe Armstrong: Why do we need modules at all? was an interesting post I remember reading a few years back. It questions the basic assumption of building software hierarchically and statically.

This is akin to the age-old (at least since birth of UNIX) desire to have programming be a set of pipes thrown together at will, with the elegance and wisdom of a seasoned hacker behind it.

What it really seems to suggest though is a move toward a way to let computers program themselves. The desire to let evolution play a key and direct role in how a computing system operates. If some given function is replaced by a more efficient function, all software depending can be faster. If you have some chain of functions `A->B->...->ZZ`, and someone writes a better chain, the whole can be replaced.

We already get some of this with packages/modules and dynamic linking, so the question (short-/mid-term) is really whether a kind of function-level alternative would replace modules/packages, or work alongside them.

## Why do we need GUIs at all?

This is a spin on the previous. It’s the basic question of whether larger elements of the GUI can be easily reused. At present the basic unit of the GUI is the widget. The widget is some semi-small piece of the GUI, like a button or a text input.

If you want a more complex GUI, it’s a matter of assembling widgets together. This is equally true for a native application as it is for a webpage or web application.

Reuse is dependent upon some human brain recognizing the opportunity, and wiring the whole thing up for reuse. That’s opposed to something like the hypothetical functions database above, which in theory would have enough metadata to be able to say, “takes in a list, outputs a sorted list, stable, memory use, complexity.”

But if you can do it with functions (which seems likely enough), you can do it with larger pieces, including GUIs. So you can have a GUI that’s designed separately from the consuming code, and it can say, “presents a list, allows reordering, sorting, adding, deleting,” and you might specify some constraints on the adding/editing portion, so it could select another GUI component (“unique (name, number) pair”, or “unique name, non-unique number pair”, or whatever).

The initial versions of this can probably be written off of `docopt`, where by the same magic of parsing `Usage` a GUI could be created for any command line API. This is a project I’ve been meaning to work on, but haven’t done it yet.

## Common theme

The common idea is reducing duplication of work, increasing productivity, removing friction to create. Making the computer figure out more of the work. Automating more. Instead of having to spend so much time doing analysis of tradeoffs and pondering why something broke when it shouldn’t, we should be building more.

That leads to the last one: moving away from pure-text programming. That doesn’t mean we write it out to binary; the text layer can persist. But it does mean that to some extent we stop doing everything as text. We can already extract interaction diagrams from text, but we currently do the editing in text and only get the diagrams when we want a conceptual view.

It will take some time before we see these last sorts of changes take place. But it’s likely that they will, if for no other reason than the benefits are there. I’d rather write the `Usage` once and know everything was wired up for me, just like I’d rather not muck about with every last bit of CSS and markup every time I want to add something to an application.

And as fun as it is to find the extra comma and find out that I wrote it correctly except for that typo, I’d rather that footgun go away entirely.