The site uses cookies that you may not want. Continued use means acceptance. For more information see our privacy policy.

What is Data Infrastructure?

In place of hot air about the definition of infrastructure, some thoughts about data infrastructure.

Data infrastructure is something like a road, or maybe a bridge. It’s a set of standards that allows data systems and consumers of data to operate without too much effort. DNS, the domain system, is infrastructure, translating readable names into numeric addresses and adding a layer of abstraction to things so that addresses can change. With paved land and spans, you don’t have to load sacks on a burro to carry goods through an untamed land, up and down hills, fording rivers, gazing at the stars and bouncing from landmark to landmark to find your way. Data infrastructure should work the same way. The modern internet needs more data infrastructure.

In email, we have the problem of spam. To deal with spam, many systems have been developed to filter it, to block spammers, and so on. It isn’t perfect, but it generally works. But we haven’t generalized that infrastructure to other problems. Some things, like anti-virus scanning of mail, uses some of the same systems, but each is generally its own thing. Other places, we see ad blockers and browsers have lists of malicious websites to warn users. Some data infrastructure exists here, but could be more generalized.

Other places, we see no real data infrastructure. Several times per year I read about journalists and others who are targets of harassment campaigns online. The social systems lack the same kind of filter technologies that email has. But it could be generalized. It should be generalized.

Identity, the ability to create a digitally-signed identity and authenticate with websites, would be a great and welcome form of data infrastructure. It has risks if it lets government censors snuff out dissent, but that already happens too often. A correctly described identity system would allow for multiple identities or multiple expressions of an identity depending on where and how it’s used.

The advertising industry already tries to create identity tokens, but users have limited control over them. Some laws get passed to try to give control, and now every website has to tell you about cookies, but you still don’t have control, you just have an extra piece of cyber garbage floating atop every website.

Infrastructure lowers friction. Building websites has a key barrier: user-sign-up. The easier it becomes for a user to sign-up, the less advantage incumbents have. That is paramount for competitiveness in many online spaces. This and other barriers are the sort that data infrastructure should break down, in the same way that the transcontinental railroad and other major infrastructure projects opened up lands for new cities and new economies.

Other commerce-related data infrastructure changes would be welcome. One is simple resource links that are platform-agnostic. They would let you link to a song without linking to a particular music service. Or link to a video game without pointing at a specific store. That kind of infrastructure helps to allow competition without forcing fans to show favorites or act as advertisers without their consent.


As the internet matures, opportunities arise to define and build data infrastructure. It took humans thousands of years to figure out roads and city planning (and we still get it wrong sometimes), but as we settle into patterns of use and behavior, and as we continue to have more data capacities, we need to be looking at ways to generalize our tools into outright infrastructure that supports the smooth flow of activities of all sorts.

What are the internet’s rivers that we can send boats down? Where are its mountains, requiring us to seek passes or blast tunnels? What will be the critical pieces of infrastructure that let computers do more work, rather than pretend to be a fancy form of paper?

Infrastructure and the Costs of Republican Opposition

Republicans are faced with a choice on infrastructure: negotiate a bipartisan bill or take responsibility for whatever Democrats pass without them.

With Democrats and the administration ready to move on infrastructure, it’s important to look at why the Republican opposition and obstruction is so damaging to legislation and to the country.

As it stands, the early posture of conservatives is that it’s too expensive to do the bill that Democrats will propose. They wish to portray Democrats as going on a debt binge (spending binge if paid for) that will harm the country in the future. Some moderate Democrats fear that attack. But perhaps it is based on true conservative convictions about spending and debt. If so, all the more reason Republicans should work to make any infrastructure bill bipartisan.

The hard truth of infrastructure is years of underinvestment, along with challenges that will only get worse including climate mitigation and reducing carbon pollution. The truth is that doing a single big bill now isn’t the solution—there is no one-and-done. We need steady and shifting investments over years as we should have been doing all along. But the political reality is that it’s very hard to get that kind of regular, orderly, sane investment done.

While the Democrats are planning tax increases to pay for the costs of their efforts, and while some tax increases and changes are needed, what’s really needed is for Republicans to take some responsibility here.

The Republicans have a choice:

  1. Try to obstruct the infrastructure efforts altogether.
  2. Work with Democrats on infrastructure.

In the former case, either they block the bills or not. If they succeed, the country suffers from underinvestment, which hurts the economy doubly. It hurts from the missed revenue going to infrastructure projects, and it hurts from the revenue and efficiencies that the infrastructure would bring.

The first case’s alternative, Democrats passing it alone, means a bigger bill, bigger tax changes, and real benefits both now and from the infrastructure changes. But it also makes it harder to have any kind of regular expectation of investment that the country really needs. And while the Democrats will do a decent job, without Republican input the results will not be as well-targeted as they could be.

In the second case, the devil and its sneakers are in the details. But it probably means a smaller bill with a different mixture of projects. It means smaller tax changes, perhaps less short-term ambition. But it still wins on the double-boost of some spending and some infrastructure gains that have long-term benefits to the economy. It also keeps the door open to a more regular cadence for investment that we should be doing.


An analogy: your family garage needs to be cleaned. If you clean it yourself, you’ll tackle as much as you can and it will be cleaner. But if your partner helps you clean it, the division of labor will change, what gets cleaned will change. In the end, the two-person cleaning will be a better job than you cleaning it alone. But neither will be as good as if you both help clean it regularly.

Most importantly, if you clean the garage solo, you might throw out or hide some things your partner will miss! The Republicans are elected to represent their states and constituents. They need to participate. If they don’t? Later, when they’re looking for the socket wrench? Tough! You had your chance.


Consider the best-case of bipartisan cooperation. The smaller bill would still be at least a trillion dollars. It would be paid for by tax changes. It would provide lesser amounts on the big targets like fixed infrastructure, manufacturing, housing. But it would still be a lot of money, in no small part because of decades of neglect.

But it would also come at a time when some of those things may be in flux anyway. Affordable housing, for example, may become more available in some cities as commercial real estate gets converted in the post-pandemic environment, assuming some permanence in a shift to work-from-home. Perhaps, perhaps not. But even if it doesn’t, it makes more sense to build housing in phases, to ensure it gets built in the right places. (It’s badly needed, but also because of the neglect it needs to be done more carefully.)

The world does change, and part of the reason for continuous investment is to adapt to those changes. It is cheaper to spend regularly than it is to spend all at once, but obstruction makes that impossible and thus drives the debt higher, the very thing opposed by the obstructions.

There will not be a perfect bill, but Republicans who try to claim whatever passes was Democrats going on a spending binge will be met with questions like: in that case, why didn’t you negotiate for a smaller bill? If Republicans truly believe these packages are too big, they are the ones in a position to negotiate. They have the opportunity to shape these bills. They should make use of that chance to try to get their own ideas out there.


There are some other reasons:

  1. Practice and talent development. If the Republicans don’t learn how to negotiate, they won’t be as effective at it in the future. If they don’t learn to legislate infrastructure, they will be in the same spot as they are with healthcare: no ideas, nothing to offer.
  2. There are moderate Republicans who don’t want to always have to point at the good things Democrats did and pretend they helped when they didn’t.
  3. The moderate Republicans will continue to be overshadowed by their rambunctious counterparts, which makes elections harder for the moderates.
  4. There are a number of ways to save money by lowering costs, which are less likely to be forced if the Democrats go it alone, meaning spending more for less.

Oh, and there’s that old dusty thing in the corner that says “right thing to do.”

A Lot is Going On for 12 October 2019

Combination post about the news from early October 2019.

The president has pulled troops back in Syria, allowing Turkey to invade and attack the Kurds. Most everyone is opposed to this, and yet the Republicans can only be so critical before they worry it will hurt their next primary. Donald John Trump is unfit to be president, and everyone knows it, but only one party can actually say it and act to do something about it.

But speaking of unfitness, you have the likes of Apple, the NBA, Blizzard Entertainment, and ESPN, all trying to appease China’s authoritarian bullshit. Not with chocolate cake, this time, but with various moves to promote their worldview, either against Hong Kong, against Taiwan, or in favor of an unfounded claim over a body of water they share with about five other countries. The preference for short-sighted profit motives does not surprise, but it does prove to be the losing side of things. The day will come when China inevitably democratizes (whether as a whole or as a set of separate states), and those citizens will remember the failure of these companies and institutions when that day comes. They will not remember them fondly.

One cannot imagine a tale of modern international history being portrayed in Chinese cinema, because they cannot tell the truth. They don’t teach it properly in schools, and they don’t portray it in their media. A society cannot be built upon amnesia. As America continues to reckon with its own past, we should recognize that other nations face that challenge, too, and worse, are far less equipped to face it. That amnesia is exactly what these corporations are supporting, for a buck.

There’s no power for chunks of California while the power company out there tries to decide if it’s safe to turn back on. Rather than do sufficient work to clear fuel on the front end, they’ve decided it’s more efficient to not make money for days at a time and hope that the weather changes. It seems better than having major fires, but one would suspect that clearing the fuel around lines would be the easier and saner way to go.

And that’s not even getting to the ongoing impeachment saga. The president continues to deny that he’s accountable for anything. The White House even sent a letter saying that impeachment was contrary to the law. One wonders, given all the times Donald John Trump has received beautiful letters from Kim Jong Un, why the White House never sends beautiful letters (re: impeachment or otherwise).

New revelations continue to come out about various worries officials have had from other phone calls and incidents. One suspects before it’s all over, Trump will claim Clinton won the election and it’s all her fault for not contesting the vote and taking the job. He already tried to pin the blame for the call on Secretary of Energy Rick Perry.

On the other hand, we’re finally having another Democratic debate this Tuesday (15 October 2019). Will be good to see the gang back together. Will be good to see Senator Bernie Sanders back at it. If the Republicans could find it in their hearts to ever have this kind of sanity in a group running for their nomination, it would be a great blessing for our nation.