Thoughts on the initiative to upgrade UIPA.org

Last update: 11/06/2024
Back to home
Back to Posts

Abstract

How does one go about upgrading a working website? While many strategies are available, selecting one that will get the job done is difficult because "you don't know what you don't know". Advice is plentiful and readily available but it's of no use for the same reason.

The kind folks of Code With Aloha have settled on upgrading by switching to the current version of Froide because it works with Django 4.2 and Python 3.10. Both are currently supported.

I think that upgrading from the old version of Froide that the production UIPA.org site is built from is a better approach because it follows the guidelines of:

"Keep it simple."
"Less is more."
"What problem are we trying to solve?"

Background

In 2016, I got involved with Code for Hawaii and did a tiny amount of work on one small part of the UIPA.org website.

UIPA.org is a website where citizens can request information from a government agency. The website stores the requests and helps with the interaction with the agencies. This is known as a freedom of information portal. In Hawaii, the freedom of information law is the Uniform Information Practices Act (UIPA), HRS Chapter 92F, which requires open access to government records. Hence the name UIPA.org which the Public First Law Center runs. It has some information about requesting public records which describes the process of requesting records.

The software for UIPA.org was built upon the open source Froide portal which is used by Frag den Staat, the German Freedom of Information Portal.

UIPA.org went live in September 2018. It was a time of Python 2.x and Django 1.10.

From then to 2024, Code for Hawaii went through some changes and became Code With Aloha but UIPA.org's software was never touched after being launched. The person that created UIPA.org dropped out and prefers not to be involved again.

The Public First Law Center's executive director, Brian Black, the new organizers of Code With Aloha met about getting the software updated. It's now the time of Python 3.13 and Django 5.2. There is a newer version of Froide with new features and dependencies. In other words, lots of moving parts to account for that weren't there before.

What to do? So, how can UIPA.org be brought up to modern times?

Code With Aloha has an initiative called A new and improved UIPA.org on their UIPA.org Development Board.

This post is about exploring a different choice. I believe that it's better to keep things as simple as possible. Let's not add more complexity and unknowns what is already a lot of unknowns.

A path of discovery

I tend to agree with the Zen of Python that simpler is better and explicit is better. The problems that I've had writing or fixing software have to do with not knowing what's supposed to be working. I use to ask one of my project managers, "What does done look like?". That would always make her angry, like "Don't you know?".

Of course not! If it's not spelled out explicitly, you're leaving too much to chance and almost guaranteeing failure.

I experienced this when I first became a software developer. At the end of the first interview with a client (a manager position) about what the scope of work would be, he asked me when it would be done. He had just rattled off a high level description of what the new system was to do.

Of course, I couldn't give him an answer because I didn't know specifically what would be needed. He wasn't able to provide much more detail. "Shouldn't he have known better since he had been in IT before?", I wondered to myself.

Since then I've been trying to figure out how to make sure that I can give the client what they need and get it to them in a reasonable time.

The way forward is like an explorer trying to get to some destination that no one's been to before. This is what software development is really like because when you want to modify code, most of the time you're hoping that it works. You're also hoping that it's going to get you where you think you should be.

My goal with this post is to describe an alternate path and report back whether or not that path reached the intended destination, a new and improved UIPA.org.

Initial forays

One of the first things that I try to do with existing software is to try to run it. With modern version control systems like git, this is fairly easy. Just clone the repo and follow the instructions if any. Of course, things don't always work; not even the tenth time or ever. There are various reasons but I'm not going to talk about that.

Assuming things do run then you're faced with what problem are we trying to solve?. Often, what someone says is the problem isn't detailed enough to clearly state what done looks like. You need to know the differences are between need, should, and want.

Upgrade Django

I tried to see if the UIPA.org code be run with Django 4.x instead of 1.10.2. Didn't work. I got an error that I couldn't resolve at that time. Definitely a show-stopper.

Make a roadmap

I quickly read through the release notes for the Django versions from 1.11 to 4.2 and made a roadmap of what things might change or pitfalls that I might encounter.

Just upgrade to the next version

OK, take things one step at a time. I tried to see if I could fix things and get Django 1.11 to work with UIPA.org. Didn't work. There was that same error that I couldn't figure out and web searches didn't help. I gave up and didn't dive deeper into the code.

However, several months later, I finally figured out how to fix that error and have upgraded to Django 1.11.17 with Python 3.8.

It seems to work for Code With Aloha

Tyler Chong with Code With Aloha was able to get a new version of Froide working but he didn't know that UIPA.org is based on using a Froide Theme and not directly on Froide. He was able to make that change so that what's in the uipa repo runs with something that looks what a new and improved UIPA.org would look like. So that's the starting point of the initiative.

Oh, they're apps. Not!

For those of you who aren't familiar with Django, a Django website is usually made up of one or more apps where an app provides a reusable amount of functionality. This should allow you to have modular pieces that you can mix and match and reuse for new websites.

The newer version of Froide has more functionality than the old one that UIPA.org uses. The newer version has features like campaigns, GEO locations, payments, and more. However, I'm skeptical that going with the newer version is a good idea because it's not clear what should be done with the new functionality. Should they be hidden? Should they be removed?

I tried removing one new functionality by changing the Django INSTALLED_APPS configuration. Didn't work. The new functionality was mixed into existing apps. You can't just leave one out because the other ones may depend on it.

Another thing is the idea of a themeable Django app. This means that you pull in another Django project as a library dependency and your project overrides the look and feel of the dependency. Froide has a Froide Theme that is supposed to be used as the project to build your website on top of. The Froide Theme then installs Froide as a dependency when you run pip install -r requirements.txt.

For Hawaii, the form provided by the Office of Information Practices has a section where the requester can ask for a waiver of the fees for the office preparing the information for the request. To handle this requirement, Froide was installed as an editable dependency and its code modified to change the way that information requests are made by a user.

While Froide is broken out into what looks like separate apps and must be configured like apps, that doesn't mean that you can leave any out of the INSTALLED_APPS list and things will still work. It's not modular, mix-and-match.

Back to the beginning

So, where does one start?

In Alice in Wonderland, Lewis Caroll writes:

"If you don't know where you want to go, then it doesn't matter which path you take."

The beginning?

The beginning is "what problem are we trying to solve". Well, that's easy! A new and improved UIPA.org! Not enough detail.

Newer versions of the software bill of materials (SBOM)? That's not enough.

We need to determine the desired functionality.

What does it do?

Instead of talking about implementation details, let's consider what a citizen wants UIPA.org to help them get done. This is typically referred to as the functionality. Traditionally, they are captured as the functional requirements.

There are various tools and techniques for gathering the functional requirements such as user stories, user mapping, process mapping, etc.

I think the most basic technique is to list out what a person does when they are trying to get something done. That means identifying:

What are they trying to get "done"?
What pieces of information are they working with?
What steps do they need to take?
What things does the system do when the user isn't directly interacting with it?

For websites, it's good to have a sitemap. In its simplest form, this is a list of pages (helps to have a name and short description of what it is or does) and the connections between them. Typically, it's represented by a drawing of boxes and lines that connect them from top (home page) to the lowest (or farthest away from the home page) pages.

One of the problem we have is that there is no list of functionality.

So what now?

It would very helpful is someone (or more people) make that list of what can be done with UIPA.org. This would help with building a test suite to help tell us what done looks like.

The alternate approach I'd like to investigate starts with modifying the old UIPA and Froide software to work with the new versions Django and Python instead of jumping to a new version Froide. I have a roadmap for this alternate path. I'm going to pursue it while the others follow the mainstream.

I plan to build tests for each of the items on the functionality list. As I go, I'll update the status of things on the roadmap and write more posts about my experiences. At the same time, I will also help out the mainstream effort.