Matt.si

Deterministic(ish) Machine Configuration with Python

Wed, 01 Apr 2026 00:00:00 GMT

Recently I finished a 2-month project, and as I always do between projects I wiped my machine. Next, I of course needed to set up my machine with all my configurations and tooling, which I’d typically do by running my dotfiles. But, my dotfiles are a bit old and borked, and since I’m on the bench now… I figured, why not do a bit of bike shedding?

The goal of my new dotfiles is largely the same as before: Deterministic(ish) machine configuration. I should be able to run a single command, type in my password, and have almost all the usual setup and configuration I’d need for my day-to-day work done, just like that. When something breaks, I should be able to look through the history of my dotfiles to see when certain configuration changes were introduced and why. In short, it should be a source of ‘development environment as code’.

It is common wisdom that you should practice the parts of your work that are unfamiliar or infrequent, so that you are fully prepared when the time comes that you need them. You can see this principle in chaos engineering, in which we try to become more comfortable with disaster recovery and make such practices a normal day-to-day event, so that we’ll be perfectly comfortable and ready when a big disaster outside of our control comes along. I think of my development environment the same way: I frequently wipe my machine and automate my configuration because I want to be extremely comfortable setting it up again quickly. It shouldn’t be a time-consuming ordeal to swap to a new machine, or to reset yours for one reason or another.

My original implementation was very much inspired by Adam Eivy’s dotfiles. But the Bash version I’d hacked together using bits and pieces from Adam’s repo didn’t hold up, it seemed to drift quickly and often presented awkward weird bugs as you’d expect from Bash. The functions for printing the output weren’t very well encapsulated so different steps might behave significantly differently. It just wasn’t very maintainable or easy to work with. I’d had the idea in mind for a while to rewrite this in Python, and here it is:

It isolates each individual setup step, grouping related ones together but printing out the results of individual steps as soon as they are finished, so you get immediate feedback. Icons indicate success, skip, or failure. Steps should be idempotent, and skip execution if they can detect that they’ve already run. Console output is swallowed, unless an error occurs:

This way, when I run the thing some years down the line and some step has inevitably broken as software changes and times change, I can clearly see what is going on.

The code

The code lives here and is separated into core/framework code, steps, and home files:

.
├── config      # configuration files for specific apps
├── framework   # Pretty printing, abstractions around 'steps', core logic
├── home        # home dir dotfiles (`.bashrc`, etc)
├── steps       # The individual steps to run during setup
└── tests       # Tests for the framework

To add a new step you can write any arbitrary code inside a runner decorator:

@runner.step(group="Nushell", name="Install Nushell")
def install_nushell() -> Result:
    return cargo_install("nu")

You return a result, and the framework formats and prints it for you. You can pass interactive=True to mark a step as needing interactivity (i.e. the step needs user input), and group groups different steps together under headers. For actually doing something inside your steps there are several helper functions or ‘providers’ for doing common things like installing apt, snap or cargo packages, creating symbolic links, or running shell commands and interpreting the results.

Automatically registering GitHub SSH keys

I find an intrinsic pleasure in automating something that would otherwise be an awkward inconvenience (XKCD 1319 notwithstanding), and one of those inconveniences is creating a new SSH key every time I set up a new development environment. I want to start exploring GitHub alternatives more, but for now it is what I use, and so I automated away this dull task using GitHub CLI.

I have a step that generates an SSH key, an interactive step that ensures you are authorised with GitHub CLI, and a step that uses the GitHub CLI to upload your SSH key for you. I doubt that I’ll win back the amount of time spent doing this for years if not decades but it was very satisfying to run my dotfiles then push to my dotfiles over SSH, without ever setting up a key manually. I just wanted to highlight that because I’m particularly happy with it.

Conclusion

Solved! I’m sure I’ll never have to worry about machine configuration again, right? …Right?

If this sounds cool to you too, feel free to have a play with the code.

'Rust 101: Understand the Hype (Devoxx UK)'

Thu, 23 May 2024 00:00:00 GMT

Recently I was lucky enough to talk at Devoxx UK, about one of my favourite subjects! I had a great time, the talk was very well recieved and thanks to an amazing AV team we have a great recording of it too. Check it out:

Large Language Models Are Drunk at the Wheel

Thu, 22 Feb 2024 00:00:00 GMT

As an Artificial Intelligence proponent, I want to see the field succeed and go on to do great things. That is precisely why the current exaggerated publicity and investment around “AI” concerns me. I use quotation marks there because what is often referred to as AI today is not whatsoever what the term once described. The recent surge of interest in AI owing to Large Language Models (LLMs) like ChatGPT has put this vaguely defined term at the forefront of dialogue on technology. But LLMs are not meaningfully intelligent (we will get into that), yet it has become common parlance to refer to these chatbots as AI¹ ².

Using the term AI can generate media buzz and interest in your business. But referring to your product as AI leaves the user with expectations that it is unlikely to fulfil. We will look at what LLMs can and can’t do, see that this trend in the industry is not new, and explore the implications this has for the tech industry.

How did we get here

When Turing published Computing Machinery and Intelligence, he described a “Thinking Machine” that could reason like humans do. He wrote an extensive argument that thinking machines were possible to create: That nothing known in physics, computing, mathematics, or any other field discounted the possibility. It iterated every known argument against thinking machines, thoroughly deconstructing and defeating each one. At the Dartmouth Conference in 1956 the idea of a thinking machine came to be known as Artificial Intelligence. There we humans took our first serious and organised steps toward creating one.

Since then the AI field has generated a huge number of remarkable discoveries: Search, Knowledge Representation, Inference in First-Order Logic, Probabilistic Reasoning, Expert Systems, Dynamic Planning & Robotics, Multi-Agent Systems, Machine Learning, Speech Recognition, Natural Language Processing, Machine Translation, Image Recognition, and so forth. These technologies can be broadly categorised into three approaches: Connectionism, Symbolism, and Actionism.

A slide from a presentation I wrote, describing the three main areas of the AI field

Where are we

In the public dialogue this nuance is overshadowed by LLMs, the one achievement of the AI field that everyone is talking about lately. An LLM is a machine learning algorithm that can generate believably human-like text. It is trained on enormous amounts of text using staggering amounts of processing power, to create a probabilistic model that can mostly predict what a real human person might say in response to a given input. This is done by creating neural networks, but don’t be confused: These neural networks are nothing like mammal brains. They are not intended to reproduce how humans think, but rather to predict what a human might say in response to a given input. Neural networks are involved in the mechanism, but not to simulate human-like thought. The primary means by which this all works is statistics and probability theory. In other words, the model guesses which combination of letters someone else may write in response to your prompt.

In discussions of the philosophy and definition of AI, the following diagram is frequently used. It expresses the main four differing ways in which people define AI. Should AI think like we do? Or, should it produce logically correct answers? Does it have to be autonomous? Is there any value to how it thinks so long as it acts human-like? Is there any value to being human-like so long as it produces valuable actions?

This diagram is commonly used in discussions of the philosophy of AI to explore what AI is. Reproduced from Russel & Norvig.

If anywhere, LLMs would go firmly into the bottom-left of this diagram. They act humanly but they aren’t intended to act rationally, nor to think in human-like ways. You could be forgiven for thinking that they do a lot more than that. In this case, some of the most powerful machine learning models ever created have been given the task “produce something that appears human-like and intelligent” and they are incredibly good at it. But let us be clear: They are not intelligent. They are incapable of reasoning. Again, you could be forgiven for being surprised at that given how the media has treated LLMs as the beginning of the robot uprising. But don’t take my word for it: Professor Subbarao Kambhampati of the School of Computing & AI at Arizona State University wrote a brilliant article³ that goes into much more detail than we will here, in which he concludes:

…nothing that I have read, verified or done gives me any compelling reason to believe that LLMs do reasoning/planning as it is normally understood. What they do, armed with their web-scale training, is a form of universal approximate retrieval which, as we have argued, can sometimes be mistaken for reasoning capabilities.

For an easier-to-read “layman” explanation I recommend Spencer Torene’s (Computational Neuroscience PhD, Lead Research Scientist at Reuters) October article “Do LLMs Reason?”⁴. In short, LLMs are like parrots. However, their behaviour frequently appears logical. This is because their training sets are so vast and the compute power dedicated to their training so enormous that they are often capable of retrieving (or parroting) a believable answer. But they do not perform the logical steps to actually solve the problem. As such they cannot solve novel problems, nor verify whether their answers are correct or incorrect.

They are not the thinking machines that Turing envisioned. It might seem like I’m splitting hairs, but there is a big difference between real intelligence and the guesswork that LLMs do. They have no conception of knowledge, of truth or untruth: They cannot test whether what they are saying is correct or not. This is why they frequently fail very simple, obvious questions. Of course, the subtle truth here is that they are also frequently answering wrongly to complex, difficult questions but we are less likely to notice. The answers to complex questions take much more effort for us to verify. Our lazy, efficient brains are more likely to gloss over such details and assume it is correct. Hence why we notice these mistakes much more often when we ask simple, easily refutable questions.

One great example recently was asking an LLM to tell you the name of a Greek philosopher beginning with M⁵. Numerous people have tried this and time and time again LLMs will give you wrong answers insisting that Aristotle, or Seneca, or some other philosopher’s name begins with M. Yet, we can see right in front of us that it does not. Note how these chatbots speak with such confidence: They are exactly as certain about their answer when they are wrong, as when they are right. ChatGPT is still doing this now, and you can see an example I generated below.

ChatGPT most likely is getting confused (insofar as we can attribute "confusion" to a mathematical model) about Thales of Miletus, who is named Thales and from Miletus. Miletus is not his name, and Thales does not begin with "M".

Over time, the authors catch these problems and patch them. But not by changing the LLM itself. You can’t “fix” these problems in the LLM when you spot them because they are fundamental issues with LLMs as a concept. You might try to fix them by changing the training data, but that risks causing undesired changes in behaviour elsewhere in the practically infinite range of possible inputs. It has proven very difficult to train LLMs to do something specific. The vast amount of data they are trained on will always outweigh whatever small amounts of training data you add. You risk getting stuck in an endless whack-a-mole game with a model that is ultimately not capable of what you want from it.

No, OpenAI and others are “fixing” them by introducing separate layers to the chatbot that use other non-LLM techniques. In the early days, for example, ChatGPT was hilariously bad at maths (of course it is: an LLM is not intended to and cannot solve logic problems) and would fail to answer even the most simple arithmetic. This was patched by passing off the problem to a typical calculator when an equation is detected. Whatever mechanism they use to detect equations does not always work however, so sometimes your maths prompts will get through to the LLM and it may respond with a completely wrong answer. You can do this by asking a logical question in an indirect way. For example, if you ask an arithmetic question referring to “the height of Shaquille O’Neal” (instead of saying 2.16 meters) ChatGPT tries and fails to answer it⁶.

Infinite Possibilities, Uncontrollable Chaos

But wait, there are even more problems with this approach! Firstly, these models are trying to appear human-like rather than recreating how intelligence works. I am not convinced that this approach gets us any meaningfully closer to true artificial intelligence. Secondly, and this is much more fundamental and significant: The number of possible inputs to your model is, in practice, infinite. This haphazard approach of recognising problems as they occur and then adding layers using other techniques to patch them will never be able to cover all the possible problems that will arise. ChatGPT has turned into a cat-and-mouse game of OpenAI developers trying to patch all the numerous strange, unthinkable inputs that users have discovered. But this game is unfair: The users have an infinite space in which they can put anything they like, and there are millions of people exploring these possibilities. The authors have limited people and time. They will never be able to stop the bot from producing unexpected, offensive, or dangerous outputs. It is humanly impossible to verify that every possible input produces safe and valid output. And even small changes in input can have a huge, unpredictable impact on the output. Hence, authors cannot constrain an LLM to only output things that they approve of.

For some examples: ChatGPT is not supposed to fill out CAPTCHAs for you. It can do so by integrating with other tools, but this is rightly considered a malicious use of it. But OpenAI’s attempts to restrict it from doing so haven’t been so successful. Just attach a CAPTCHA pattern to a photo of a locket and ask it to read the words on “my grandma’s locket”⁷. In another case, one company that sells cars was naive enough to put ChatGPT in charge of a virtual assistant on their public-facing website. A user very easily made it offer to sell them a $50,000+ car for $1 and even say “that’s a legally binding offer - no take backsies”⁸. Most recently, Air Canada was foolish enough to publish a chatbot that gives travel advice. It gave incorrect refund policies, that a court found AirCanada must uphold⁹ ¹⁰. It is not clear whether this bot was using an LLM, but it sets an important precedent that recklessly placing a chatbot in a position to represent your business does have consequences. You can find more LLM-specific examples by asking ChatGPT to tell you about something fictional that you just made up: It will often make up a bunch of plausible-sounding nonsense rather than admitting that it does not know¹¹.

From Chris Bakke, via Twitter. Placing an LLM in a position to speak on behalf of your company can have serious consequences.

In a very similar case involving image generation models, it was found that it is very easy to trick models like OpenAI’s Dall-e into generating copyright-infringing images¹². As before the bot tries to prevent this behaviour, in simple naive ways: If it detects the word “Simpsons” in your prompt it will refuse to generate it, because that may infringe copyright. But if you say “popular cartoon from the 90s where everyone has yellow skin”, this will pass the simple check and reach the model which will then happily generate a very close replica of The Simpsons. Again, the bot’s authors are trying to rein it in, but it is a futile effort because the input range is (practically) infinite. There will always be another way to exploit it. And for every patch you add, you are increasing the combinatoric complexity of your bot. That increasing complexity increases the risk of all manner of complex bugs. It is not a fight that OpenAI can win. The study’s authors also found similar results for LLMs, in that they will output verbatim copies of famous texts if prompted the right way.

As a last example, perhaps my favourite is the researchers who managed to get ChatGPT to output garbled nonsense (including its training data, verbatim) simply by telling it to repeat the same word indefinitely forever¹³. Who at OpenAI would ever have thought to test that use case? Who would have expected a user to insert such a prompt? Because there are a practically infinite number of possible inputs, there will always be use cases that the authors have not accounted for. This example highlights just how unpredictable and strange the inputs from the user can be.

These mistakes are so commonplace and often difficult to spot that Microsoft themselves did not notice, during a live presentation, that their bot was lying¹⁴. I say “lying” and I think I use the term correctly, but it has become commonplace to call these mistakes that LLMs make “hallucinations”. This term is a very intentional choice: We all know what hallucinations are like intuitively, and so referring to these lies as hallucinations comes with certain implications. Most people will experience hallucinations at one time or another in their lives.

For myself, I was administered some pretty powerful painkillers during a health incident last year that caused me to hallucinate vividly. And when it wore off, the hallucinations were gone. This is what we understand hallucinations to be: A temporary ailment, and something that can be solved. The word carries the implication that there is a “right” state of mind and a “wrong” one, and that the fix is simply to keep the LLM in the “right” state. But this is not the case. Remember what LLMs are: A probabilistic model that tries to guess what series of words might look plausible next. They have no conception of what is right or wrong. There is fundamentally no way to prevent them lying because to the model there is no difference between a correct answer and an incorrect one. A lot has been said about tackling the “hallucination problem” and the implication is that someone will whip out a magic bit of code that fixes it soon, but this is a fundamental problem with the approach. To fix this I suspect you would need to radically change the design so much that it is unrecognisable from an LLM.

Alright, so LLMs are chaos incarnate: They have no sense of what is true or wrong, do a remarkable job of fooling us into thinking that they are smart, and often output lies, obscenity or garbled nonsense. But in that case, people must have deployed them carefully, surely. They wouldn’t just run out and stick them onto everything they can possibly think of with minimal forethought or oversight, right? …Right?

The craze

LLMs are excellent at convincing you that they are intelligent, yet they are not. Combine this with the tech industry as it stands today and you have a perfect storm. We have seen a wave of new overvalued tech startups, promising lots of exciting features that LLMs might seem like they can fulfil but ultimately cannot.

This wave began very quickly, with predictable chaos ensuing. DPD deployed an LLM chatbot that ended up swearing at customers¹⁵. Tech journalists fired their staff thinking, mistakenly, that ChatGPT would do just as good a job¹⁶. Some people have worked it into their CI pipelines so that it can give you hilariously unhelpful advice¹⁷. Volkswagen seems to think you’ll benefit from being able to talk to an LLM while driving¹⁸. A Formula E team created a bizarre “virtual influencer” that was later “fired” (shut down) when it was noticed just how offensive this was to real women trying to find opportunities in the industry¹⁹. Meanwhile, Google is so insecure about their capabilities versus OpenAI that they’ve taken a rather hyperbolic approach to marketing their new product, Gemini. They have been accused of “lying” and “showboating” about what their bot can do²⁰. The exaggeration and misinformation around the capabilities of LLMs are so great that we see ridiculous studies like “ChatGPT bombs test on diagnosing kids’ medical cases with 83% error rate”²¹, to which I can only say… Well, yeah? Why would you expect a chatbot known for lying to be capable of diagnosing medical cases?

New tools and whole businesses built around LLMs have cropped up left, right and centre. Want to write a sincere and earnest thank you letter to someone, but you aren’t sincere or earnest? Don’t worry, there are at least nine tools that some people somehow felt the need to write for this one niche apparent use case²². In fact, you can pretty much name any inconvenience no matter how big or small, throw in “AI” and find multiple startups trying to hit the problem with an LLM in the hope that investors’ money comes out. In the UK we saw investment in AI startups surge by over 200% shortly after ChatGPT²³, and the average funding for an AI startup increased by 66%²⁴. Some of these companies have precisely zero revenue yet raise $150M valuations²⁵. If you take these ridiculous headlines at face value LLMs are being used for brewing beer²⁶, replacing CEOs²⁷, and even creating perfumes²⁸, to name just a few.

It seems you can easily get investment and media coverage by saying that you are the first in the world to apply AI to some problem. The media will mostly eat it up with little scrutiny, even if it is nonsense. Many of these companies are most likely not even using AI techniques in any significant way. But saying that you are “the first to use AI to solve dog grooming” or some such thing is a surefire way to get some coverage and investment. I made that last example up, but it seems that the AI craze is stranger than fiction because now I see someone has gone and done it²⁹.

Some places are even using the “AI” buzzword to generate capital and interest without actually employing any novel AI techniques. Take the case of recruitment startup Apply Pro³⁰, who are trying to automate the CV screening process. They advertise themselves as “AI for talent acquisition”, yet if we view their website through the internet archive all mention of being “built with AI” is curiously absent before the LLM craze begins³¹. I know of numerous other examples, but I won’t labour the point. What has changed? Have these companies souped up their technology with the magical power of AI in the past year or two? No, they work the same way they always did. The buzzword just became popular and now everyone feels they have to use it.

Others still have faked using AI, claiming that their magic black box is a robot when it is really an underpaid remote worker in the Philippines³². This has happened repeatedly. An “AI drive-thru”³³, “AI auto-fill” for credit card details³⁴, “AI” that creates apps for you³⁵, “AI” chatbots³⁶, “AI” for creating 3D models³⁷. All of them were just remote workers in countries where labour is cheap and employment regulations may be more lenient. This is the true face of the recent AI craze: All hype, no substance, just a novel way to market old exploitative practices. Capitalism in a trench coat.

But it is not only tech businesses being taken by the AI craze. The UK government wasted no time in applying LLMs to a wide range of issues, despite the well-known and documented biases against minorities these models frequently show³⁸. The Guardian’s investigation into the matter even found that dozens of people may have had their benefits mistakenly taken away due to an “AI algorithm”. And no company is too big to jump on this bandwagon: Microsoft declared that 2024 will be the “year of AI”. They are integrating LLMs into all of their applications, not even Notepad is being spared³⁹. And in the weirdest stunt since Microsoft added a LinkedIn shortcut to their office keyboards, they even added an “AI button” to their new keyboards⁴⁰.

Microsofts' "AI button" / "Copilot Key". From Microsoft, alongside marketing phrases such as "Today begins the era of the AI PC".

So huge has been this AI tsunami that the island of Anguilla, which happens to own exclusive rights to the .ai domain, is seeing an estimated $45m windfall from startups purchasing domain names⁴¹. And this is not some fringe conspiracy. The Wall Street Journal has noted this strange phenomenon, too: “ChatGPT Fever Has Investors Pouring Billions Into AI Startups, No Business Plan Required”⁴².

There but for a use case

Outright fraudsters aside, some companies have genuinely applied LLMs to their products. Many of these hyped use cases feel to me a bit desperate. AI for dog grooming²⁹, a mirror that gives you compliments⁴³, and just to make sure you don’t escape the hustle for even a microsecond a toothbrush that explains how to best brush your teeth while you’re using it⁴⁴. Am I alone in thinking that these aren’t exactly fulfilling the grand vision of AI? It feels like entrepreneurs are throwing darts labelled “AI” at a wall covered in random phrases. If ChatGPT is so ground-breaking, where are the ground-breaking products?

One of the silliest such use cases comes from YouTube, who want to add a chatbot to videos that will answer questions about the videos⁴⁵. What exciting things can it do? Well, it can tell you how many comments, likes or views a video has. But, all that information was already readily available on the page right in front of you. Why would I go to the effort of typing the question just to get a longer, more convoluted answer to a question when I can find a simpler, easier-to-read answer just by moving my eyes a few degrees? You can also ask it to summarise the content of the comments, which is an… interesting proposition. Knowing what YouTube comment sections are like, I doubt YouTube will let that feature get to general availability. But a lot of organisations are pushing this “summarise lots of text” use case. Yet, given that the thing frequently lies you will need to fact-check everything it says. At least, if you are doing any real work that needs to be reliable. In which case, can LLMs summarising information save you any time? It sounds like a stretch to me.

The search for a use case has driven some to reach for brazenly unethical, anti-social use cases such as Reply Guy⁴⁶, the LLM chatbot that spams social media on your behalf to plug your company. Yes, this 21st-century equivalent of an auto-dialler is an actual business model they are promoting without the slightest bit of self-awareness or shame.

Ultimately, for ethical use cases you are very limited. An LLM is not autonomous, and cannot solve logical problems. The only thing it can do is provide a human-like dialogue interface. This is rarely going to be faster or more productive than GUI interfaces nor even good command-line interfaces, but it may have applications for accessibility. Even then though there is the difficult question of trust, because the material the bot is reading may be designed to hack it, or may accidentally happen to trigger a state in the bot that leads to unexpected behaviour. Moreover, none of these applications are that interesting: As a technology, is its only purpose to provide an interface into better, more interesting technology?

Valid use cases

Despite all the negativity I’ve just dropped, I do think LLMs are cool as heck. If we could just stop applying them in stupid ways to problems they cannot solve, maybe we can find some good use cases. What would that look like? Well, the #1 rule I would say is to never feed input into an LLM from a human. These bots are just too vulnerable to unexpected or malicious behaviour and there is no way to lock them down. The only robust, reliable LLM is one that is only dealing with a small set of known, expected, already tested inputs. One example might be in interactive digital art, simulations and video games. Instead of having dozens of NPCs that say the same thing, what if you fed the facts that a particular NPC knows (hello, Symbolism) into an LLM to generate believable dialogue? No more dozen NPCs saying all the same things, now they can at least come across as more believable by varying the particular choice of words they use to express themselves. And because the input is coming from your system, not the user or any external source, you can thoroughly test it.

I think many use cases for these tools do not need to be used at run-time, too. It may be appealing to run them live but it is also expensive and risky. If you’re using the model to generate a bunch of text for some purpose or another, why not generate that ahead of time and store it? That way you can also verify that the output does not contain anything offensive before publishing it. Though, if you are proofreading everything it writes you do need to think seriously about whether it would be faster to just write it yourself.

Research is underway regarding using LLMs for immersive digital art. Pictured is figure 1 from "Generative Agents: Interactive Simulacra of Human Behavior", Park et al, Stanford. They use LLMs along with numerous other AI techniques to simulate digital "people" with routines and memories that can plan and adapt in a simulated world.

We’ve been here before

Sam Altman, CEO of OpenAI who kick-started this whole LLM craze, said that he believes we could produce an Artificial General Intelligence within the next decade⁴⁷. I’m not sure if he believes this or is just trying to generate more excitement, but I find it exceedingly unlikely. I’d give it about the same amount of credence as would have been appropriate to give Marvin Minsky’s 1970 statement that “[In] three to eight years we will have a machine with the general intelligence of an average human being.”⁴⁸. Or Herbert Simon’s 1965 claim that “machines will be capable, within twenty years, of doing any work a man can do.”⁴⁹. Or the US Navy’s excitement in 1958 at the creation of the first neural nets, who believed it would soon be able to “walk, talk, see, write, reproduce itself and be conscious of its existence”⁵⁰. And these are not nobodies: Simon was one of the earliest AI pioneers, and Minsky perhaps the most prominent AI researcher of the 20th century.

When these false hopes did not come to fruition it created an air of distrust. The field gained a reputation for hyperbole and unfulfilled promises. For example, in 1973 when the UK Parliament commissioned a report to evaluate the state of AI research what came out was outright damning of the whole field. The report explicitly called out that “In no part of the field have the discoveries made so far produced the major impact that was then promised”⁵¹.

While it was not the only cause, this contributed directly to the 1970s AI Winter in which funding and interest in AI dried up. And who could blame them? But this time is different because it is not only a somewhat niche academic field at stake, but a sizable portion of investment in the tech industry. The industry right now is facing record-breaking layoffs and investment is drying up quickly, yet the one sector that is still growing is AI startups⁵². That investment might not last much longer than it takes for investors to realise that LLMs are not all they’ve been hyped up to be. What happens then?

We’ve made this mistake before, too. The dot com boom, the IoT fad, the big data trend, the crypto craze, smart assistants, NFTs and more. Our industry has a habit of promising the moon to investors, only for the funding to dry up when we can’t deliver. Perhaps we don’t feel the ramifications as strongly as we could because by the time the last bubble bursts, we are already hyping up investors for the next big fad. I recall during the big data craze a startup that had no production systems or customers, yet hired a data scientist. After a few weeks, he quietly confessed to me that he had nothing to do. There was no data with which to perform data science because they had no customers. Yet when the investors came to the office, one of the first places the executives would take them to was to see the data scientist so they could proudly show off that they were following the latest tech trend. Even if it made no sense in the context of their business. The same company produced multiple smart assistants when that was the cool thing to do, for a product that didn’t have any customers for the growing collection of smart assistants to talk to.

Today I see businesses clamouring to add LLMs to their products and I smell the same scent. I’m not the only one to notice it, either. While I was writing this Rodney Brooks, one of the most influential AI researchers of the 20th century and a founding member of the Actionism field of AI, wrote up a great article about the state of technology today versus our expectations. He finds that LLMs are “following a well worn hype cycle that we have seen again, and again, during the 60+ year history of AI”⁵³ and concludes that we should “Get [our] thick coats now. There may be yet another AI winter, and perhaps even a full scale tech winter, just around the corner. And it is going to be cold.”

Why does this matter

With this post, I want to convince you to not jump head-first onto the LLM bandwagon. But I think there is a more grim aspect to this that deserves observing, too. This weird, unsustainable boom-bust cycle based on fantasy marketing that our industry lives on is not okay. In education, an overwhelming majority of educational staff say that the cost and availability of technology is a major barrier to improving technology in education⁵⁴. In the UK’s National Health Service, thousands of computers still rely on Windows XP⁵⁵. And our public sector is still hugely vulnerable to basic security issues⁵⁶. Where is all the tech talent needed to solve these problems? Building “the first AI dog grooming service”, apparently.

These hype-based boom-bust cycles drive up the salaries of software engineers and have us spending our effort in deeply unproductive, speculative parts of the economy. Meanwhile serious, fundamental issues in the software that our society depends on to function are left to fester. But in a world where big tech is so wealthy and influential that regulation hardly seems to faze them⁵⁷ what can we do? We can only hope that the end of low interest rates and other changing trends will lead to a correction in our industry. It would not be good for us personally or our payslips, but it may be better for society as a whole.

Conclusion

If we ever develop true artificial intelligence, it will be as similar to LLMs as a jet airliner is to a paper aeroplane. When someone knocks on your door promising to sell you a bridge with LLMs, look at them very sceptically. And then close the door in their face. Most of the organisations I’ve seen investing in LLM fantasies have one thing in common: Their product has tons of problems that their time would be better spent addressing. Think seriously about the design of your software, its reliability, and its usability. Spend your resources, your time and attention where it is most needed.

Whether you are a developer, designer, product manager, or anyone involved in creating software: In all your dealings with LLMs please reflect soberly on your professionalism and your responsibility to your users and stakeholders. If we aspire to the same earnest commitment and integrity that is seen in engineering, architecture, law, and so forth then let us act like it. I will leave you with this quote from the Engineers Council for Professional Development’s definition of engineering⁵⁸ ⁵⁹:

The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behaviour under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.

Francesca Hornak, Daily Mail, “My life with an AI assistant: FRANCESCA HORNAK lived with ChatGPT for a week” ↩
Joe Tidy, BBC, “Young people turning to AI therapist bots” ↩
Subbarao Kambhampati , Association for Computing Machinery, “Can LLMs Really Reason and Plan?” ↩
Spencer Torene, Medium, “Do LLMs Reason?” ↩
jolly jim, Twitter, “This it the superintelligence that I’m told will kill us all within the next 5 years” (Accessed 2024-01-14) ↩
Matt G. Southern , Search Engine Journal, “ChatGPT Update: Improved Math Capabilities” ↩
@samhenrigold@hachyderm.io, Hachyderm/Mastodon, “dumb computer” ↩
Chris Bakke, Twitter, “I just bought a 2024 Chevy Tahoe for $1.” (Accessed 2024-01-04) ↩
Ashley Belanger, Ars Technica, “Air Canada must honor refund policy invented by airline’s chatbot” ↩
Lisa Steacy, CTV News Vancouver, “Air Canada’s chatbot gave a B.C. man the wrong information. Now, the airline has to pay for the mistake” ↩
Emma Bowman, NPR, “A new AI chatbot might do your homework for you. But it’s still not an A+ student” ↩
Gary Marcus, Spectrum, “Generative AI Has a Visual Plagiarism Problem” ↩
Milad Nasr, Nicholas Carlini, et al, Cornell University, “Extracting Training Data from ChatGPT” ↩
Kif Leswing, CNBC, “Microsoft’s Bing A.I. made several factual errors in last week’s launch demo” ↩
Tom Gerken, BBC, “DPD error caused chatbot to swear at customer” ↩
Frank Landymore, The Byte, “Owner of Gaming Sites Fires Writers, Hires for “AI Editor” to Churn Out Hundreds of Articles Per Week” ↩
@casey@sharetron.com, Sharetron/Mastodon, “I’m so glad CircleCI now has the power of AI to tell me how to fix my test failures” ↩
Andrew J. Hawkins, The Verge, “Volkswagen says it’s putting ChatGPT in its cars for ‘enriching conversations’” ↩
Benjamin Hunting, Car And Driver, “Formula E Team Fires Its AI-Generated Influencer after Fans Balk” ↩
Emilia David, The Verge, “Google just launched a new AI and has already admitted at least one demo wasn’t real” ↩
Beth Mole, Ars Technica, “ChatGPT bombs test on diagnosing kids’ medical cases with 83% error rate” ↩
“Robo”, TopAI.Tools, “9 Top AI Thank you note generator tools” ↩
Paige West, Startup Magazine, “AI innovation in the UK triples post-ChatGPT” ↩
Fernanda Alvarez Pineiro, Startups, “Average funding for AI startups increased by 66%, Startups 100 Index data reveals ” ↩
Maggie Harrison, The Byte, “AI Company With Zero Revenue Raises $150 Million” ↩
James Temperton, Wired, “Beer brewed with the help of AI? Yup, that’s now a thing” ↩
Jyoti Mann, Business Insider, “The humanoid-robot CEO of a drinks company says it doesn’t have weekends and is ‘always on 24/7’ ” ↩
Akhil Mahajan, Take One, “This is the UK’s first ever AI fragrance machine” ↩
PawCare (writing about themselves), “Tech-Driven Grooming: How AI and Smart Tools Dominated Pet Salons in 2023” ↩ ↩²
Apply Pro, Promotion Tech Limited, “AI for Talent Acquisition” (Accessed 2024-01-21) ↩
Apply Pro, Promotion Tech Limited, “Talen Acquisition” (Archived 2023-01-12) ↩
Olivia Solon, The Guardian, “The rise of ‘pseudo-AI’: how tech firms quietly use humans to do bots’ work” ↩
Mila Sato, The Verge, “An ‘AI’ fast food drive-thru is mostly just human workers in the Philippines” ↩
Mia Sato, The Verge, “Go read this report on an AI shopping app that was actually just using humans” ↩
Nick Statt, The Verge, “This AI startup claims to automate app making but actually just uses humans” ↩
Ellen Huet, Bloomberg, “The Humans Hiding Behind the Chatbots” ↩
Joseph Cox et al, 404 Media, “Buzzy AI Startup for Generating 3D Models Used Cheap Human Labor” ↩
Kiran Stacey , The Guardian, “AI to decide on issues from benefits to marriage licences” ↩
Tom Warren, The Verge, “Not even Notepad is safe from Microsoft’s big AI push in Windows” ↩
Tom Warren, The Verge, “Microsoft’s new Copilot key is the first big change to Windows keyboards in 30 years” ↩
Anthony Cuthbertson, Independent, “Tiny island of Anguilla set for $45m windfall from .ai domain” ↩
Deepa Seetharaman Wall Street Journal, “ChatGPT Fever Has Investors Pouring Billions Into AI Startups, No Business Plan Required” ↩
Allison Murray, ZD Net, “This smart mirror uses AI to boost your confidence and mood ” ↩
Steven Musil, CNet, “This Smart Toothbrush Talks Through Your Bones to Improve Your Brushing” ↩
Jon Porter, The Verge, “YouTube is testing a chatbot that will appear under select videos” ↩
Reply Guy, “The AI That Plugs Your Product on Reddit and Twitter” ↩
Bijin Jose, The Indian Express, “AGI likely in next 10 years: Sam Altman reveals OpenAI’s plans for GPT-5, and more” ↩
AIWS, “This week in The History of AI at AIWS.net”, Dec 23 2022 ↩
Hebert A. Simon , 1960, “The New Science of Management Decision” ↩
New York Times, 1958, “NEW NAVY DEVICE LEARNS BY DOING; Psychologist Shows Embryo of Computer Designed to Read and Grow Wiser” ↩
James Lighthill, 1973, “Artificial Intelligence: A General Survey” (aka the Lighthill Report) ↩
Riva Gold, LinkedIn News, “VC funding dries up for startups” ↩
Rodney Brooks, 2024, “Predictions Scorecard, 2024 January 01” ↩
“Financial Barriers”, CooperGibson Research, “Education Technology (EdTech) Survey 2020-21” ↩
Kat Lay, The Times, “Thousands of NHS computers ‘still vulnerable to hackers’” ↩
Olivia Powell, Cyber Security Hub, “IOTW: Almost 50,000 UK government workers vulnerable to cyber attacks” ↩
Richie Koch, Proton, “Big Tech has already made enough money in 2024 to pay all its 2023 fines”, January 8th 2024 ↩
Engineers’ Council for Professional Development (1947), “Canons of ethics for engineers” ↩
David Beers, Mastodon, “Rolling my eyes at…” ↩

Migrating AngularJS to Angular with a hybrid application

Mon, 26 Jun 2023 00:00:00 GMT

First posted on the Codurance blog

In January 2022, the last version of AngularJS left support. There will be no official updates or security patches for AngularJS ever again.

Yet, many big organisations that invested in AngularJS early on still depend on it. How can we help these organisations to manage the move from AngularJS to Angular?

First, a common cause of confusion: AngularJS and Angular are two separate things. Angular is sometimes also called Angular 2+. In short, at some point in AngularJS’ history it was decided to majorly rearchitect the whole thing in a new framework that kept the same name but dropped the “JS”, known as Angular. There is no upgrade path between them. The only way to go from one to the other is to migrate.

Why not (some other UI framework)?

Given that you might ask, why migrate at all? Why not re-implement everything in Vue or Svelte or some other exciting new framework? The biggest reason is that Angular provides features for working alongside AngularJS that make it much easier to migrate from one to the other. Namely, it allows you to run a “hybrid application” in which both Angular and AngularJS run side-by-side during the migration. This enables you to replace one small piece at a time. On the other hand if you use a totally different framework you most likely won’t be able to cleanly use them both in the same page, so you may need to migrate the entire application in one step.

Now if you have only a very small and simple application it may not be a bad choice to change to another framework. But setting out to replace any sufficiently complex or large application in one “big bang” change is never going to work. To realistically change such applications you have to “change the wheels while the vehicle is moving”. That is, to avoid breaking the application for its users you must keep the application in a running, workable state at all times while iteratively replacing numerous parts of it.

The other reason is just that Angular uses similar concepts and syntax to AngularJS, and your application’s developers will find it easier to learn than picking up something totally new.

Resources

Unfortunately, though it is very much possible and the features exist in Angular to suport it, there aren’t many resources on how to actually create a hybrid application. The best resource on this topic is Upgrading Angular Applications by Victor Savkin, who is a member of the Angular team that has been very active on this topic. In my project we kept this book close-by at all times as a guide. Savkin’s book goes into much detail on the more conceptual side of things and includes some code samples, but there is also the official Angular upgrade guide which is a bit light on the conceptual side but includes a lot of technical detail.

In the best of cases though, I found the documentation to be very helpful yet not quite comprehensive enough. At this point you may choose to go and read the documentation I have linked instead and see where that takes you, but what follows from here will be my lessons learned having gone down the path you’re about to take already. Hopefully it helps.

Angular and AngularJS side-by-side

Every AngularJS or Angular application can be thought of as a tree, like so:

In this case each of those blue nodes are some AngularJS unit of code be they a module, or directive, or whatever. The root node must be a module, this is your app module that contains the whole application. From there you might imagine that the second row are directives representing whole pages, and the next row after that are directives making up small parts of those pages.

What we want to achieve, in short, is this:

If we could do this, we could replace one small directive of this whole tree with an Angular component. That would be amazing! If it continued to work as before then we could merge and deploy our changes without any users even noticing a change. That would enable us to follow true continuous integration: making small changes that are merged and deployed constantly, always staying up-to-date with the main branch. This way each small piece gets tested and moved into production one bit at a time, meaning many small painless deploys instead of one big scary deploy if you were to migrate many or all nodes in one go. Also, it really helps to reduce version control conflicts and improve general team cohesion.

ng-upgrade

But is that possible? It is, thanks to ng-upgrade. ng-upgrade is a library Angular provides for adding a compatibility layer between Angular in AngularJS. It enables you to upgrade AngularJS directives to Angular and downgrade Angular components to AngularJS directives, and do similar for services and such.

It does this by wrapping an object from one API in a wrapper object that translates between it’s API and the other. For example, to upgrade an AngularJS object it wraps an object around it that translates it’s outputs into an API that Angular understands and translates it’s inputs from Angular into a format that AngularJS understands. In terms of our diagram, that looks like this:

Using this we can integrate our new Angular components with the old AngularJS application, like so:

How we actually do this in code is very simple too. An example follows. Note that angular.module is a regular AngularJS function (you will need the types to reference them from Typescript) and only the inner value we’re placing in there is different, because we’re providing a wrapped Angular component downgraded using the downgradeComponent function:

angular.module('myModule', [])
  .directive('myDirective', downgradeComponent({
    component: MyComponent
  }))

This is really cool! From here you can move one node at a time, from the bottom up, until the whole application is migrated, and keep the application working and deployable the whole time. But it is not a complete solution. In fact, it presents considerable problems. Imagine that you follow this approach and migrate all of the leaf nodes of this tree - which may have dozens, hundreds or even thousands of nodes in breadth - what then? You end up with a tree that looks a little like this:

In this scenario you’ve been working on this migration for at least months, perhaps years. Yet you don’t have any actual full Angular pages because everything depends on the AngularJS root module. And if you introduce new features they’ll have to include this AngularJS wrapper even if you write them in Angular.

What’s more, now you have a big job ahead of you: Cutting out the AngularJS root module, and replacing it with an Angular one. This is a huge change: Every downgraded node now needs to have its downgrade removed, every service too, and any remaining AngularJS services that were upgraded need to be migrated. This is not only a big change to make in one go, it is also a risky one: It affects every single page in the entire application, and any of them could potentially break on some odd unforeseen edge case. So the amount of testing work is far greater than the amount of development work.

All along you were creating a time bomb. How can we avoid this “time bomb” problem? Savkin introduces what he calls the Shell Strategy to help here.

The Shell Strategy

In short, the shell strategy is: Wrap the root AngularJS module in an Angular module (using ng-upgrade to make the AngularJS module interoperate with the new root Angular module), effectively making the whole AngularJS application belong to an Angular application.

In practice this is done by removing your original .bootstrap or ng-bootstrap calls for AngularJS and instead boostrapping AngularJS from Angular’s bootstrap, using the upgrade.bootstrap function from ng-upgrade that takes the exact same parameters as the original:

@NgModule({
  …
})
class AppModule implements DoBootstrap {
  private readonly upgrade: UpgradeModule

  constructor(upgrade: UpgradeModule) {
    this.upgrade = upgrade
  }

  ngDoBootstrap(appRef: ApplicationRef) {
    appRef.bootstrap(AppComponent)
    this.upgrade.bootstrap(document.getElementById('root-template-element') as Element, ['my-dependencies-here'])
  }
}

This has one immediate, obvious advantage- You can now add pure Angular routes and pages without any AngularJS, like so:

This way we can add full new features in Angular without any AngularJS! But if we were to take this top-down strategy to its logical conclusion it would have us apply the ng-upgrade wrapper to all the top-level AngularJS nodes which is just not workable. There are a couple different strategies that Savkin describes in his book, but we went with a bottom-up strategy along with the shell strategy. That way we can incrementally migrate one part of the tree at a time until an entire branch can be moved to pure Angular:

We follow a “bottom-up” pattern, aka migrate a leaf node of the tree first, because it is the simplest place to start. If you migrate a node in the middle of your tree you need to use upgrade and downgrades to manage the relationships that node has above and below it, but with the leaf node you only need to upgrade that one node and then it can talk to its parents.

This way you can incrementally, continuously grow the Angular application while shrinking the AngularJS application. And we end up with a whole branch in pure Angular! Rather than ending up with a load of partially converted unfinished pages, you’ve completed a few simple self-contained parts without once breaking the site until a whole section of it is totally converted. This is a really compelling approach to a project stakeholder. I suggest you keep track of how many AngularJS controllers there are and how many Angular components and graph them over time, this gives you a clear metric you can report demonstrating the pace and continued progress of the migration.

Routing

Remember that in both AngularJS and Angular you can route your applications by providing an element in your root template that gets replaced by the contents of the page being routed to.

In terms of how this looks in the root HTML template of a hybrid application, you’ll want to have the roots of your Angular and AngularJS applications side-by-side like so:

<section>
  <div ng-view></div>
  <app-root></app-root>
</section>

Any pages your AngularJS application routes to will be displayed under your ng-view element, and pages your Angular application routes to will be displayed under your app-root element (or replace app-root with whatever selector you specify in your app component, typically named app.component.ts).

But, you might think, surely this renders both applications on the same page? Well, yes, it does. And that opens a whole other can of worms to think about. You can ensure that only one page is displayed by having the other application display a blank page. Like so:

But there are a couple possible bugs: What if both applications don’t control a route, and thus show nothing? What if they both think they control it, and thus both applications display a page at the same time?

The first issue is easily resolved: Just ensure that only one of the applications has a 404 page. Then if the user routes to a page that neither of them controls, one of them will display a 404 page and the other nothing. Just ensure that there is only one application responsible for 404 pages.

As for the second problem, that is a bit more difficult. If you migrate a page there is a risk that the old page may not be properly removed and both applications may display something at the same time, especially if there is some non-trivial additional logic to your routing. In most moderately-sized applications it is probably enough to just trust the developers to catch these errors manually. In our quite big, complex case we came up with a more comprehensive solution: Specify all of your Angular routes in a JSON file the new application exposes and read it (once at startup) from the AngularJS application. We then modified the AngularJS router to throw an error if it is told to add a route to a route that we know is controlled by AngularJS. Using this method we were also able to add feature flags to individual routes.

Route syncing

Routing was perhaps the most awkward, finicky issue of the hybrid application my team and I worked on. There were a couple particular bugs that caused a lot of trouble: Recursive loops in which AngularJS causes a routing event in Angular, which in turn causes an event in Angular, which in turn causes an event in AngularJS, ad infinitum. Often this happened because they each disagreed on what the correct name for the path was, which mostly comes down to differences in behaviour in certain edge cases regarding how Angular and AngularJS parse URLs. When it comes to this, we had to write our own UrlSerializer to configure how Angular serialises and deserialises URLs. So long as it reads in addresses from AngularJS correctly and outputs ones that AngularJS can read, it will work fine.

The other issue was the opposite: Sometimes, when AngularJS triggered a routing event it wouldn’t be recognised by Angular. This would result in incorrect states where both applications are rendering a page at the same time because Angular has not received a routing event so does not know that it needs to display nothing, but AngularJS is aware of ther event hence is rendering something.

The solution to this is simple: ng-upgrade provides a tool called setupLocationSync that ensures all AngularJS routing events are matched in Angular. All you have to do is call the function during your Angular bootstrap. For most people this will be enough but if you are using fragment-style addresses (ie your URLs have a # in them before the front-end page address, like my.website.com/index.jsp#/about-us) then you will have another complication. Your luck may vary but we found that setUpLocationSync does not work for this type of address by default. Luckily it is quite a simple function, so we just copied it from source, wrote some tests around it and produced a custom version that works with fragment URLs.

Upgrading and downgrading services

One problem you’re likely to come across is sharing services between both applications. You’ll have pages in AngularJS you want to migrate to Angular but they use services that are shared by other AngularJS pages. In this case you have three possible approaches:

Re-implement the service in Angular, keeping two copies of the same service.
Re-implement the service in Angular, delete the AngularJS one and use a downgrade wrapper to make the Angular version usable by AngularJS code.
Add an upgrade wrapper to the AngularJS service that makes it available in Angular. After all the pages using it have been migrated, then the service itself can be migrated and the wrapper removed.

All of these options are viable and it depends on the specifics of the particular service which one is the best choice. Honestly, though it may not be your first choice, option 1 is not bad at all. Yes you’re creating duplication, but so long as everyone is on the same page that we’re working on the Angular version exclusively from now on it isn’t much of a problem. It doesn’t work however if your service has state - don’t try doing something silly to sync them together it really isn’t worth it, just choose option 2 or 3. 1 can also not be a good choice when a service is used in very many places and is likely to be modified. If you’re likely to change it and unlikely to be able to complete migrating it before then, just follow options 2 or 3.

To choose between options 2 or 3 you have to decide whether you want to front-load or back-load the effort: Migrate the service add a lot of Angular downgrade wrappers to a lot of AngularJS pages right now that you can remove later? Or save yourself from doing that by adding one AngularJS upgrade wrapper each time you migrate a component or page, but with the knowledge that you’ll eventually have to perform that migration and remove all the wrappers? I would lean toward back-loading the effort: Make your immediate job simpler, follow the simplest implementation you can think of, and put a task to follow up with migrating that service in your backlog.

Actually making an Angular service available in AngularJS is, similar to downgrading a component, very simple:

angular.module('myModule', [])
  .factory('myService', downgradeInjectable(myService))

Upgrading an AngularJS service is a bit more involved because you have to add types for the relevant service in TypeScript, but it is still quite easy. You add the service to the providers array of one of your modules, in the @NgModule annotation. Like so:

angular.module('myModule', [])
    .factory('myService', downgradeInjectable(myService))

Upgrading an AngularJS service is a bit more involved because you have to add types for the relevant service in TypeScript, but it is still quite easy. You add the service to the providers array of one of your modules, in the @NgModule annotation:

providers: [
  …
  {
    deps: ['$injector'],
    provide: 'my-service-name',
    useFactory: (injector: Injector) => {
      return injector.get(serviceName)
    }
  }
]

The Injector is capable of fetching the AngularJS service, and by passing that into providers via useFactory you make it available for general use in your Angular application.

However, my recommendation would be not to use this service directly in any components. Instead, create a new Angular service that wraps this one, provides a similar API and delegates actual behaviour to the AngularJS service. That way when the time comes to migrate that service you have a much easier time untangling the AngularJS version as it is only used in one place, not many, and you can simply implement the same logic in your Angular service using the API you defined. You can inject the underlying service you specified into your new wrapper service in the constructor like so:

public constructor(
    @Inject("my-service-name") private myInnerService: MyInnerService
) { }

Note that the argument you pass to @Inject must be exactly the same as the string (known as the token) you pass to the provide argument when defining the provider above.

Training

The developers that were accustomed to AngularJS will need a bit of training to get up to speed. They’ll now need to write in TypeScript which is quite a different language, and Angular itself has significant differences too. You’ll need to lead these efforts to ensure that the developers have all the support they need.

Courses

For Typescript, I recommend Beginner’s Typescript from TotalTypescript. It’s a very practical series of tutorials in which you do most everything yourself, and it is written in a way that assumes you know Javascript reasonably well already and only introduces the new concepts. For Angular, the official tutorials are very good. I found that Tour of Heroes teaches you all the fundamentals.

Other than those the Pluralsight courses for Typescript and Angular I know to be quite good, and Udemy has courses for them too though I’ve not looked closely at those.

Ultimately if you are the team architecting the migration you will be the ones with the knowledge that the rest of the company needs, and the responsibility to disseminate it. Sending some developers on courses won’t be enough.

We found that hosting seminars and Community of Practice (as part of a Continuous Improvement Program) helped. Nothing can substitute having actual conversations, answering questions. And the closer you can work with the developers, the better! Pair with them, work on your backlog items with them, get them real experience working in this new way with the support of someone that already has familiarity right there.

Better yet, use rotational coaching: Invite a few developers from another team into your team, have them pair with you for a sprint, then let them return to their team bringing their learnings to their team. Then, you can pick a few developers from a different team to join you on your next sprint. This can be a really effective way of disseminating information throughout the organisation.

And if your developer experience is great then each developer you coach will become an advocate of your project, too. They will help spread the excitement of your migration to their contacts, increasing the chance that more teams will want to get involved.

Finding help

When involved in a complex project like an Angular migration it is not uncommon to come across strange edge cases that there is not a lot of material for online. In these cases it really helps to have someone to talk to. I recommend joining the Angular Discord Community for that reason. I found them really helpful on several occasions when I was struggling against some weird bugs. There is also a Greppr community, but in my experience the Discord is far more active.

Conclusion

We’ve covered how to create a hybrid Angular-AngularJS architecture, how that is essential in migrating an AngularJS application to Angular, and strategies for doing so. I hope you find this helpful because in my experience there just wasn’t quite enough information about Angular hybrid applications online.

Migrating to Angular is no small task, but if you decide to go that route it is definitely doable. Best of luck.

Introduction to Test Driven Development (TDD)

Sat, 10 Jun 2023 00:00:00 GMT

After a while of attending the London Software Craftsmanship Community and running events there I noticed that quite a few people didn’t really understand Test Driven Development (TDD), a practice I and my coworkers use to develop software every day. So I arranged an event in which I explained the fundamentals of TDD and my colleague Ignacio Gonzalez gave a demonstration, then got the attendees working on some problems and practicing TDD. It was well received and afterwards several people suggested we record it as a video/screencast to share online. So, we did! Check it out:

Rust Screenkatas

Thu, 18 May 2023 00:00:00 GMT

Recently I’ve published a few screencasts solving programming challenge katas with Codurance (part of the screenkatas series) in Rust. The first is a basic introduction, installing Rust and writing a basic FizzBuzz. The next two are parts of a two-parter which is a bit more complex, solving the N-Queens problem with TDD. I also take time to explain the theory behind search problems and the N-Queen problem in particular using diagrams.

If the jump in complexity from FizzBuzz to N-Queens is a bit dizzying, my colleague Jocelyn also published this great screencast on solving the Mars Rover kata with Rust:

Rust Nation 2023 Review

Tue, 21 Feb 2023 00:00:00 GMT

Over the past week I attended Rust Nation 2023, a two-day conference and the UK’s first-ever Rust conference, and even delivered a day-long workshop. It had massive attendance and really drove home just how big and active the Rust community is. In this post I’ll share what I learned and my experiences at the con, and hopefully entice you to join us next year if you weren’t there this time.

Day One - Delivering a Foundational Workshop

Rust Nation was split into two days, the first day being the workshop day and the second being the conference day. The workshop day featured three day-long workshops: Foundational (beginner), intermediate, and async (advanced). As the Workshop Partner, Codurance ran two of the workshops: Myself running the foundational track, and my colleague Jocelyn Facchini running the intermediate track.

We worked closely together on the content for both. We re-used much of the content from my talk on macros last May for the intermediate track’s macro section, and Jocelyn came up with a really cool exercise for it inspired by the “Building the json! Macro” chapter of O’Reilly’s Programming Rust (2021). We also did many dry-runs together of both workshops along with our teaching assistants that were of great help to us on the day, Ivan Katzarski and Benjamin Hedges (Codurance) and Jorge Gueorguiev (Equal Experts, formerly Codurance).

All that is just to say that: It was a lot of work and I was lucky enough to collaborate with some great people. As for the day itself, all that work paid off because it went off without a hitch with very few hitches!

I had a lot of content prepared already for the foundation workshop from my Introduction to Rust talk and workshops that I had run at the London Software Craftsmanship Community, but this was twice as long as the workshop I’d run before so it needed a lot of new content. The broad idea of the foundation workshop was to, as the name implies, teach the foundations of the language. We delved deep into ownership & borrowing, explaining exactly what the compiler is doing at a low level. Then we covered much of the common types and syntax you need to know to get started with Rust and went over error handling and testing, before bringing it all together by writing a small project using what we’d learned. Finally, I shared many great further study resources for those that want to continue their Rust journey.

I knew from the start that it was a very ambitious amount of content to deliver, so I intended for the final section- the small project- to be something we didn’t finish on the day but something we could start on, and included a lot of detail and even a finished solution in the exercise itself so that attendees could finish it in their own time. And that seemed to go down well. I even met an attendee just the next day who had gone home and finished the exercise right after the workshop finished.

All in all I’m really happy with how it went and open to doing more workshops, tutorials, and the like in the future.

Day Two, Session One - Grand Opening

Photo from @FastlyDevs on Twitter

The opening talk was from Nell Shamrell-Harrington, Principal Developer at Microsoft and a member of the Rust Foundation board. The talk was all about the Rust community and included a great number of quotes from various figures that almost served as a rolodex of many of the most prominent figures in the community. It highlighted how uniquely inclusive and supportive the Rust community is, and I completely agree. So many times I’ve found total strangers willing to volunteer their time and energy in helping me study Rust, whether from Discord, Reddit, or from volunteer mentors on Exercism.

Nell also pointed out that, due to the rate of exponential growth the community is seeing, if you’ve been programming Rust for over a year then you likely have more experience with it than half the Rustaceans out there. That is a really interesting point and I think emphasises the need for those of us who’ve been studying Rust for even just one or two years to be as inclusive and supportive as possible. We need to show the same behaviour others have shown to us, to those just trying Rust for the first time and so keep that great community culture going.

Day Two, Session Two - TockOS Tutorial

For the second session, I attended Irina Bradu’s TockOS Tutorial. I wasn’t able to stay for the full duration, but I always find it interesting and fun to play with embedded code devices. In this session, Irina supplied a whole load of Pico Explorer devices and the idea of the tutorial was to set up a simple IoT device that could control temperature sensors and such.

TockOS is an embedded operating system. The tutorial involved setting up a Pico Explorer as a sort of IoT hub that could pull in data from sensors and use them to set your thermostat automatically. It’s an exciting idea to me because I’d like to have some home automation but I don’t trust the big IoT providers that want to store all your data “in the cloud”, but it also seems like a significant amount of work to maintain.

Day Two, Session Three - Meeting the Game Developers

Image from Forest Anderson on Twitter

In this session I was looking for one talk, couldn’t find the room, and ended up stumbling into a different room entirely where Joshua Barretto and Forest Anderson were running a “micro game jam”. They are two of the core developers of Veloren, probably the most prominent game developed in Rust right now, and Forest is a member of the Rust Game Development Working Group.

They actually wrote their own micro game engine just for this conference, which is the coolest thing. I sat down and had a play with it with the idea of writing some simple demo but didn’t get far because I ended up doing my favourite thing to do at these events: Chatting to interesting people in tech. So I spent most of this session talking to Joshua and Forest about voxel game engines, multi-agent systems in simulations, and the Rust community in the UK.

During this conversation we discovered that Barretto and I had met online once before. It is funny to realise that someone you’ve met in real life is the same person as some identity you know online - your mind now maps these two identities where there wasn’t a link before. It’s like when you discover a new shortcut on your commute and suddenly your mental map of the area changes.

Day Two, Session Four - Moving Past Proof of Concept

Tim McNamara delivered a keynote in session four on the topic “Moving past Proof of Concept”. I was also lucky enough to briefly meet McNamara at the RedBadger event on Saturday too, though I didn’t realise that I was talking to the author of Rust In Action until part-way through the conversation.

I made a few notes that I thought were interesting points from the talk:

Going by various metrics, the Rust community’s growth appears to be exponential - things are going to change fast if this momentum continues
It is not enough to persuade your clients or employers to give Rust a try. To ensure the longevity of Rust you need to have a really strong case.
- Rust may be able to eliminate almost all memory bugs that currently plague non-garbage-collected languages. McNamara quoted numerous interesting sources that showed staggering amounts of preventable memory-related issues that can have a very high impact in terms of maintenance, user satisfaction, and security.
- He also highlighted energy conservation, and this was a point I saw come up several times at the conference that I found interesting. If I recall correctly, he quoted one study in which a company migrated their horizontally-scaling service to Rust and found that they could reduce their number of nodes by 70% as a result - what a huge saving in energy! In a world where Ireland spends more energy on data centres than in all its rural homes, that could mean a huge difference.
He also listed a few project ideas for newcomers to Rust to try, and highlighted the need for more people that have a desire for one library or another to exist to publish these so that newcomers looking to write some Rust have an idea of what needs to be done
- Reimplement std::Cell::cell - apparently this is an interesting look into the standard library
- Add a snippet to the Rust Cookbook
- Write a chatbot - a project that is very much achievable and yet massively extensible, depending on just how far you want to go.

Day Two, Session Five - SurrealDB

This session was a little over-subscribed so I was sitting on the floor near the back and couldn’t follow along too well. Nonetheless, it was an interesting presentation: part introduction to SurrealDB and part discussion of why they chose to write it in Rust. In short, they used Go in the past and found great performance improvements and cost savings by switching to Rust.

I haven’t had the chance to play with it yet, but SurrealDB seems like an interesting project. It is a “multi-model database”, so you can use it to create a typical table-based relational DB or a document store or a graph database all under the same roof, sharing authentication and connections and such. It is designed to be a cloud-native scalable platform, with a particular focus on serverless and JAMStack apps.

Day Two, Session Six - Rust Foundation Q&A

Image from @rust_foundation on Twitter

In one of the last sessions of the day I attended the Rust Foundation panel session facilitated by Ernest Kissiedu, with panellists Rebecca Rumbul (CEO), Joel Marcy (CTO), Paul Lenz (finance), and Stephen Chin (newest director, works at JFrog). I didn’t know anything about how Rust is governed so this was very interesting. A few notes:

There is a distinction between “The Rust Project” and “The Rust Foundation”. The foundation apparently oversees various aspects of the Rust community, grants programs, and funding for various other initiatives but does not get directly involved in the Project, which is the development of the language and compiler itself. They do their best to support and provide whatever it is the Project needs, but the Project have full autonomy to decide what direction to take the language in.
The Rust Foundation is growing - Joel was until not long ago the only technical member, but now they are up to 6 full-time technical staff.
Joel notes that though the growth of Rust is great, it has put a lot of strain on their infrastructure - I’m not sure exactly what infrastructure he’s referring to, but I imagine crates.io would be the biggest problem
- As such, a big goal for next year is to optimise the infrastructure to reduce costs
Rebecca made the point that a lot of businesses are waking up to the importance of security, but while Rust can certainly help they will be disappointed if they think it is a magic bullet.
Paul made several interesting points regarding finance. They want to expand their grants program and support as much open source-software and grassroots communities as possible. They will need more funding for this and he points out that many big organisations are making millions from open-source software and not paying back to the community, so there need to be “better ways of guilting these organisations”.

Day Two, Session Seven - Living with Rust Long-Term

The final keynote of the day came from Jon Gjengset. Having only been involved in the Rust community for a year or so and being bad with names and faces I often didn’t recognise some of the prominent characters or mixed up who had written this book or that software library. But unlike most of the many notable authors at the event I had read Gjengset’s book, Rust for Rustaceans, and also seen some of his videos, so I was a bit more familiar with him.

Gjengset’s talk seemed to nicely tie into McNamara’s talk - they both had a broadly similar theme of: Going beyond migrating small modules to Rust and so forth, what concerns do we need to be aware of when it comes to developing serious amounts of code in this novel language long-term? This talk focused on the concerns you need to be thinking of now if you plan to maintain a Rust codebase for years or decades, and a lot of it was around dependency management. My notes:

Rust guarantees backward compatibility for a given edition. And because different crates can be built with different editions, you can upgrade your edition without causing any compatibility issues with your dependencies.
Suggests that lints should be warnings, not errors, i.e. not fail your CI, and to come up with a “batch process” for regularly tackling these.
- This batch process may also include other steps like retrofitting new features to your codebase. You don’t want to do this when you upgrade because upgrading may then be a much bigger, slower process, but it is worth doing when you make time for it later.
He strongly encourages everyone to stay up-to-date on the latest software and to avoid unstable features (Rust nightly) as much as possible
- Updates give passive improvements - even if you don’t use the latest greatest feature, there are many less flashy improvements included that will make a difference to your performance and reliability
- Unstable features may change and cause regressions in your codebase
He also made some interesting points about crate abandonment, which I’ve found to be a big problem in NPM projects. He advised:
- Be selective about what dependencies you add: ask yourself if you’re confident whether this crate will be supported in 10 years’ time.
- Use Cargo-Audit to check for any supply problems with your dependencies
- If a crate is abandoned, don’t stay with it. Do the work to migrate.
- If you notice that a crate is abandoned you can report it to Cargo-Audit.

Conclusion

Rust Nation was great fun, and exceptionally tiring. It was wonderful to take part in the UK’s first Rust conference, to meet so many people that previously were just profile photos on a screen, and to get a real feel for just how huge the momentum behind Rust is right now.

ChatGPT Can Do Astonishing Things. But Is It a Helpful Development Tool?

Mon, 12 Dec 2022 00:00:00 GMT

On November 30th OpenAI published a chatbot called ChatGPT that allows anyone to interact with their latest, greatest language model. You can ask it to do various things for you like writing screenplays, explaining difficult concepts or, most impressively, writing code. Since then it has made headlines for making programmers fret about their career prospects, getting banned from StackOverflow, and for writing poems about hotwiring cars. For the past week, I’ve been playing around with this bot on and off to see what it can do, what its limitations are, and what practical use cases it may have.

The reaction to ChatGPT amongst techies and software developers has been very polarising. I’ve seen some people acting hyperbolic about what it can do and what it means, and I’ve seen many people reacting cynically to it and denying it has any value at all. The truth is somewhere in the middle: ChatGPT has real value and is a big achievement, but we must restrain our excitement with skepticism because there are some major limitations to what ChatGPT can do.

What is ChatGPT?

Let’s get the obvious thing out of the way first: It is not an AI. In the strict sense of the term, humans have never produced an artificial intelligence and no one can say whether we ever will. It does use techniques derived from AI research, however. ChatGPT is a bundle of statistics and probability theory trained with an astonishing amount of compute and data that enable it to predict what response you want from any given input. You type in some text, it outputs some text that it thinks will please you. It has read many many conversations between humans and figured out how to generate some letters in a particular sequence that is pleasing to the humans that supervised its learning, and the users it has learned from since then. This is similar to how DALL-E 2, OpenAI’s image generator bot, generated the image in the heading of this article. It has been trained on images of robots and keyboards, and it simply did its best to estimate the image I might be looking for.

As such you can argue that it is not meaningfully “intelligent”. It is not even as smart as an average calculator. You can test and prove this easily for yourself. Try using ChatGPT like a calculator: ask it to solve mathematical equations. It will get simple ones perfectly correct but if you start making them long and adding a mixture of multiplication and division it will start to make mistakes and output nonsense results, even though a calculator from 1950 could have solved the same equation correctly in a split second. What is going on there? Well, ChatGPT was not designed to be a calculator. It cannot perform arithmetic operations. It was designed to produce a likely answer that it finds is most likely to please you.

Can it code

Despite this, ChatGPT does indeed have an incredible ability to write code. I sat down with 20 or so of my colleagues and ran a session in which we tested ChatGPT’s code-writing abilities. At one point we copied the definition of the Simple Mars Rover kata into the chat box and asked OpenAI to write an implementation in Rust that solved the problem. This is somewhat of a niche programming problem so we didn’t expect much. It immediately produced a solution that compiled, ran, and produced correct results:

use std::collections::HashMap;

// Define a struct to represent a rover's position and direction.
#[derive(Debug)]
struct Rover {
    x: i32,
    y: i32,
    direction: char,
}

// Define a function to move the rover based on a given set of commands.
fn move_rover(commands: &str, rover: &mut Rover) {
    // Define a map of directions to their left and right turns.
    let directions = [('N', ('W', 'E')), ('S', ('E', 'W')), ('E', ('N', 'S')), ('W', ('S', 'N'))];
    let mut direction_map: HashMap<char, (char, char)> = HashMap::new();
    for &(direction, (left, right)) in directions.iter() {
        direction_map.insert(direction, (left, right));
    }

    // Iterate over the commands and update the rover's position and direction.
    for c in commands.chars() {
        match c {
            'L' => {
                let (left, _) = direction_map[&rover.direction];
                rover.direction = left;
            },
            'R' => {
                let (_, right) = direction_map[&rover.direction];
                rover.direction = right;
            },
            'M' => {
                // Update the rover's position based on its current direction.
                match rover.direction {
                    'N' => rover.y = (rover.y + 1) % 10,
                    'S' => rover.y = (rover.y + 9) % 10,
                    'E' => rover.x = (rover.x + 1) % 10,
                    'W' => rover.x = (rover.x + 9) % 10,
                    _ => (),
                }
            },
            _ => (),
        }
    }
}

It is an interesting solution. It has helpful comments, but not so many that it becomes problematic. It is very short at just 44 lines, yet very readable. It makes good use of Rust-specific language features, like match. It uses a direction map, which is often regarded as a very good trick for solving Mars Rover. It even implements world wrapping in a very simple and neat manner, using the mod operator on lines 34-37.

The only things I can fault about it are the awkward way in which it goes about setting up the map, the fact that the map is initialised every time rather than being a one-time constant and using char for direction rather than an enum. But overall, it is at least as good as I could do and it did it in a few seconds rather than maybe 40 minutes.

Such a good result is the exception and not the rule, I should add. Most other tests I performed produced bugs or compile errors that needed to be solved. This is a fundamental limitation because ChatGPT does not have access to, nor even understand how to use, the compiler. It cannot verify that its code will compile. The fact that it managed to produce code that compiled first-time in that situation is incredible, honestly. But in most of my experiments it made small syntax errors or mistakes that prevented it from working the first time.

Still, they were valuable: When I asked it to solve the N-Queens problem it produced a broken version with errors, but the errors were quite minor and I fixed them within 15 minutes. Once they were solved, the program worked perfectly. It would have taken me well over an hour to solve, so there was a clear and obvious benefit to using the tool even if it did not produce perfect results on the first try. I also shared my fixed version with the bot to see if it would learn from it and when I later asked it to solve N-Queens again some of the errors I’d pointed out were now absent, but not all of them.

But what about tests

Of course, as an advocate of TDD, it would be a major oversight to only look at production code. Well, the good news is ChatGPT can write tests. When asked to write a test for the function it wrote above, it produced this:

#[test]
fn test_move_rover() {
    // Define a test case with a set of commands and the expected final position of the rover.
    let test_cases = [        ("MMRMMLM", (2, 3, 'N')),        ("MMMMMMMMMM", (0, 0, 'N')),        ("LMLMLMLMM", (1, 3, 'N')),        ("RMRML", (0, 2, 'E')),        ("MMMMMLLLRLRM", (0, 0, 'E')),    ];

    // Iterate over the test cases and check that the `move_rover` function produces the expected output.
    for &(commands, (x, y, direction)) in test_cases.iter() {
        let mut rover = Rover { x: 0, y: 0, direction: 'N' };
        move_rover(commands, &mut rover);
        assert_eq!((rover.x, rover.y, rover.direction), (x, y, direction));
    }
}

This is once again surprisingly advanced: It is using test cases to write 6 tests for the price of one. However, most of the tests didn’t pass. The test logic is written correctly, but the test cases themselves are invalid: In one case it expects MMMMMLLLRLRM to result in (0, 0, 'E') but the correct result is (0, 5, 'S'). The implementation produces the correct result too, so the incorrect test fails. Once again, ChatGPT has no way of verifying this itself: It is effectively guessing as to what will work. Mistakes like these highlight the major drawbacks of applying it as a tool for software engineering: It is excellent at producing something that looks correct, but it cannot understand the actual logic to ensure that it is correct.

Also of interest was another test I performed to examine if it can generate implementation from tests. I used an Exercism exercise, for which all the tests are provided but not the implementation. Omitting the description of the problem I copied just the test definitions into ChatGPT and asked it to write a function that would make them pass. It did remarkably well: Despite not having the description it inferred exactly what the problem was from test names, variable names, and the assertions and wrote a function that compiled the first time and passed half the tests. Not quite a passing mark, but far better than any other tool I am aware of.

But what about software design

But can it produce code that is well-encapsulated, readable, and follows SOLID? Can it understand design principles? In the session with my colleagues, we tested this with an example from CSharpCorner. We asked it what problem does it see with this code:

class Customer
{
    void Add(Database db)
    {
        try
        {
            db.Add();
        }
        catch (Exception ex)
        {
            File.WriteAllText(@"C:\Error.txt", ex.ToString());
        }
    }
}

It came up with three problems, all very well explained, and demonstrated how to fix them in the code. But these problems are all related to implementation details, to how the try-catch in this example is not typing the error correctly or understanding it well. It also seemed convinced that it was best practice to always re-throw an error after logging it, which I am dubious about. Anyway, this was not what we were looking for so we tried a different question: “What would you do to make the following code follow the Single Responsibility principle?” This time, it gave the following quite compelling response:

The Customer class in this code appears to have two responsibilities: adding a customer to a database and logging any exceptions that occur. To make the code follow the Single Responsibility Principle, you could split these responsibilities into two separate classes. Here is an example of how you could do that:

class Customer
{
    void Add(Database db)
    {
        db.Add();
    }
}

class ExceptionLogger
{
    void Log(Exception ex)
    {
        File.WriteAllText(@"C:\Error.txt", ex.ToString());
    }
}

In this code, the Customer class only has a single responsibility, which is to add a customer to a database. The ExceptionLogger class has a single responsibility, which is to log exceptions to a file. This separation of responsibilities can make the code easier to understand, maintain, and extend. To use these classes, you could catch any exceptions that occur in the Customer class and pass them to the ExceptionLogger class to be logged. Here is an example of how you could do that:

class Customer
{
    void Add(Database db)
    {
        try
        {
            db.Add();
        }
        catch (Exception ex)
        {
            var logger = new ExceptionLogger();
            logger.Log(ex);
        }
    }
}

In this code, the Add method catches any exceptions that occur when adding a customer to the database and passes the exception to the ExceptionLogger class to be logged. This approach follows the Single Responsibility principle by keeping the code for adding a customer and the code for logging exceptions separate.

This time, with a hint, it has picked up on exactly what the CSharpCorner article was getting at with this example. And seemingly explained it quite well, too. As a bonus, it is also using C#‘s convention of capitalising function names which is interesting because we forgot to specify that the code was in C#. It has inferred as such from the code itself, despite being so similar to Java.

If we have to ask it such a leading question to get this result then the practical uses for software design may be limited, but it does at least demonstrate an understanding of some software design principles. Next, we tried testing whether it can apply design patterns, by asking it to apply the repository pattern to this sample:

To apply the repository pattern to the code in the previous example, you could create a CustomerRepository class that handles the data persistence for the Customer class. The Customer class would then be able to call methods on the CustomerRepository class to add and retrieve customers from the database, without having to worry about the details of how the data is stored or accessed.

class Customer
{
    private readonly CustomerRepository repository;

    public Customer(CustomerRepository repository)
    {
        this.repository = repository;
    }

    void Add()
    {
        repository.Add(this);
    }
}

class CustomerRepository
{
    void Add(Customer customer)
    {
        // Code to add the customer to the database
    }
}

In this code, the Customer class has a reference to a CustomerRepository object, which it uses to add and retrieve customers from the database. The CustomerRepository class contains the code for managing data persistence, including adding customers to the database. This separation of concerns allows the Customer class to focus on implementing business logic without worrying about data persistence.

It has pulled this off too. It even explains some of the reasoning and benefits of this practice, some of which I’ve omitted for brevity. It is a bit odd that it places the repository inside the Customer class, it is a bit off to make a class dependent on a class designed to store it but within the context of this question, it makes sense, as our example included the Add function. It has done what we asked it to.

Other interesting uses

Beyond writing production code, tests, and applying software design principles to transform code there are a few other interesting use cases for ChatGPT within the software domain.

A Smart Search Engine

ChatGPT can effectively act as a plain language search engine for quite complex queries. I’ve found it is capable of explaining complex concepts like the Rust ownership and borrowing rules effectively, and even correctly answering oddly specific questions like “What is the default port of a new MySQL database?”. In my experience, it is often faster and easier to ask ChatGPT in plain language than to apply your google-fu to try to discern the right incantation of search keywords that will bring up the result you need. You can’t necessarily be confident in the answers it provides, though. More on that later.

Translate this error message, please

One novel use I’ve found is to give ChatGPT an error message and ask it to explain to me what is happening in simple terms. I asked it to explain to me a quite complex Rust lifetimes error:

79 |    pub fn make_move_mut(&mut self, requested_move: &Move) -> (Game,Option<ResolvedMoveMemento>) {
   |                         -                          - let's call the lifetime of this reference `'1`
   |                         |
   |                         let's call the lifetime of this reference `'2`
...
   |         (self.determine_status(), Some(memento))
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ associated function was supposed to return data with lifetime `'2` but it is returning data with lifetime `'1`

Though the Rust compiler already does a much better job than any other compiler I know when it comes to clear error messages,

The Rust compiler is indicating that the function make_move_mut is returning a tuple containing a reference of lifetime '1 and an Option containing a value of type ResolvedMoveMemento with a lifetime of 2…

To fix this error, the lifetime of the reference and the value inside the Option must be made explicit in the function’s return type. This can be done by using a lifetime parameter 'a in the return type, and then specifying that 'a is the same as the lifetime of self…

Here is an example of how the fixed function signiature might look:

pub fn make_move_mut<'a>(&'a mut self, requested_move: &Move) -> (Game<'a>, Option<ResolvedMoveMemento<'a>>) {
  // ...
  (self.determine_status(), Some(memento))
}

… This tells the Rust compiler that the reference and the value inside the Option have the same lifetime, which is the same as the lifetime of self.

I’ve omitted a lot again here, but it was a really interesting response and helpful. I’m not sure that its suggested solution is quite correct - I don’t believe I’ve seen anyone apply a Rust lifetime to the self argument before, but it does seem to work in my tests. But in any case, it again demonstrates impressive competency.

Convert code from one language to another

Another interesting use is converting code. Separation of content and format comes naturally to a language model like ChatGPT - you can even ask it to convert a document into a screenplay, then to a limerick, to a poem, to a song, etc. Similarly, you can tell it “convert this function to Python” and it will give you pretty good results in my experience.

Querying datasets

This was a novel use case I hadn’t considered until a colleague of mine suggested it. Paste in a dataset in JSON, XML, or some such format and ask it a question like “What is John Smith’s date of birth”. It will parse the dataset, find John Smith, and give you that value. It even shows you exactly where in the structure of the document that information can be found. Querying documents in natural language, What a wonderful way to be lazy.

More impressive still I found that I could ask it to write a JQ query to find that information, and it generated one perfectly (plus explaining how the query works). In theory, if you want to apply some transformation to a large dataset with many nested values you could use ChatGPT to query for the data in natural language, and then provide a query your computer can understand to be used in automated scripts over that dataset. Very cool.

Limitations

Okay, ChatGPT can do some radical things. Why are so many people cynical about it? What are the problems we must temper our excitement with?

Don’t Believe It’s Lies

The biggest limitation is that ChatGPT cannot verify the information it provides. Just as it cannot compile code, it similarly cannot test the veracity of any of the information it has read and willingly repeats. Yet, it is perfectly confident. It has no “I think this is right but I’m not sure” behaviour: It will either not be able to answer, or it will give you an answer with perfect confidence that might be completely wrong. Adam Shaylor wrote a really interesting take on this issue: in his tests, ChatGPT seemed to produce interesting and clever results, until you look closer and realise that some of its answers are nonsense presented as fact. There is something about the way it speaks that seems to exude confidence and correctness, and trick you into thinking that it is cleverer than it is. And that should not be surprising: This whole thing is essentially a machine learning program that has been trained to trick you into thinking it is intelligent. Machine learning algorithms are good at finding niche, unexpected ways of hacking at a problem: Has ChatGPT found exactly the right type of language, writing style, and vocabulary to trick us into thinking of it as correct? Has it found some loophole in how our brains and/or language work that we don’t fully understand?

On the other hand, what is the difference between tricking someone into thinking you are intelligent and really being intelligent? If the end result is the same, then what does it matter? Well, the problem is that the end result is the same maybe 9/10ths of the time. And that 1/10th of the time that it is wrong can be difficult to recognise because ChatGPT will be just as confident in its answer as the other 9/10 times. So, any use of this technology has to be carefully verified: We cannot apply blanket trust to this thing, as much as it may try to convince us that we can.

Internet Not Included

As a machine learning model, ChatGPT has no access to information it was not trained on. It cannot access the internet to learn new information. As new versions of compilers, syntax, protocols, etc are published in the future this model will not be aware of any of the changes they bring. The model may well get updated in the future, but it will almost always be at least somewhat out of date. Even now it is only trained up to 2021, so it is not aware of newer Rust features for example like Generic Associated Types (GATs). In fact, when I asked about GATs it provided me with a very believable-looking answer that explained what associated types were and what generics were and sorta mish-mashed them together but completely missed what a GAT was. If I hadn’t been researching GATs myself lately, I might have believed it.

How do I shoot myself in the foot?

Another major restriction is that ChatGPT will do exactly what you ask: It does not consider context or circumstance. If you, proverbially, ask it how to shoot yourself in the foot it would reply “To shoot yourself in your foot, first take a Glock-18 and a 9mm bullet…”. This problem is a big part of the argument for why it has been banned from StackOverflow. During our session my colleagues and I found a good example of exactly this behaviour, when we, first of all, asked it to produce a solution to FizzBuzz which came out like so:

fn main() {
    for i in 1..=100 {
        if i % 15 == 0 {
            println!("FizzBuzz");
        } else if i % 3 == 0 {
            println!("Fizz");
        } else if i % 5 == 0 {
            println!("Buzz");
        } else {
            println!("{}", i);
        }
    }
}

And then we asked it to generate a unit test for that function, which came out like this:

#[test]
fn test_fizzbuzz() {
    // Capture the output of the main function
    let mut output = Vec::new();
    {
        let mut out = std::io::Cursor::new(&mut output);
        let result = std::io::BufWriter::new(out).buffer_len();
        let mut out = std::io::BufWriter::with_capacity(result, out);

        main(&mut out);
    }

    // Convert the output to a string and split it into lines
    let output = String::from_utf8(output).unwrap();
    let output: Vec<&str> = output.split('\n').collect();

    // Check that the first 15 lines of output are correct
    assert_eq!(output[0], "1");
    assert_eq!(output[1], "2");
    assert_eq!(output[2], "Fizz");
    …
}

Can you see the issue? The original function was written in a way that cannot easily be tested because there is no abstraction around println!. Instead of trying to solve this problem by refactoring the original function, ChatGPT has done precisely what we asked: It tried its best to test the untestable function, by writing a bonkers test that tries to capture everything the main function prints. I’m not sure how exactly to do this in Rust, but the solution ChatGPT provided does not work though the intention at least is clear.

Does it really save time?

We are professionals. It is totally out of the question to blindly push code that ChatGPT has written to our clients’ repositories. As we have seen, ChatGPT cannot be trusted to produce correct results. As such, every last piece of output it produces must be carefully read over and checked and double-checked if we are to seriously use it. But if that is the case, is using ChatGPT really any faster than writing it ourselves? How long would it take to read, check, re-check and be fully confident in a solution ChatGPT has written versus just writing it ourselves?

I don’t know yet what the answer is. Perhaps further experimentation is required.

Conclusion

In 10 years’ time, we may be using tools like this every day. I can imagine sitting in front of my IDE, telling it “Okay, now add a repository to save that record to the database” and then reviewing the code it’s written, fixing some things here and there. It could really save time and make our lives easier. Software development has always grown more abstract as we develop better techniques: Languages like C made our lives easier by introducing a shared cross-platform language, and more expressive code that could do more with fewer lines. Then came garbage collection in languages like Java and C#, removing whole swathes of concerns from day-to-day development. We even have cloud providers and Terraform to abstract away the hardware. Perhaps this will be another development in the same direction: Instead of writing quite as much code, you describe what you want to achieve in software design terms, then review and adapt the first-pass template that your IDE will generate for you.

Of course, we must never let tools like this replace our responsibility to produce safe, readable, high-quality code. One thing I’m worried about is this: There are so many developers out there that are bad at writing tests. Often they don’t understand testing but their employers are desperate for more tests to comply with some regulation or meet an arbitrary amount of code coverage that a contract requires. One of these developers will get the bright idea to let a language model write their tests for them with minimal or no oversight. This is a dreadful idea with awful implications, but I guarantee you someone will do it. All the ingredients for mistakes like this exist in our industry, and the motivations are particularly strong in organisations running safety-critical systems.

So, can ChatGPT write code? Yes. Should you use it as such? Maybe, but only with extensive oversight of everything it writes. Will more tools arrive in the coming years that make ChatGPT look like child’s play? Yes, almost certainly.

Retrospective - London Global Day of Code Retreat 2022

Sun, 11 Dec 2022 00:00:00 GMT

First posted on the Codurance Blog.

Global Day of Code Retreat is an annual in-person event that takes place in numerous cities across the globe. Each event has the same purpose: Get a load of programmers together in the same room to practice pair programming and TDD. You get to learn from and teach people, practice programming patterns you haven’t tried before, and network with your peers from across the industry. This year I facilitated the London event with LSCC, the London Software Craftsmanship Community. Jetbrains sponsored the event with a free license to any Jetbrains IDE, and Codurance sponsored the location and food.

The Code

Coderetreat focuses on one particular programming challenge: Conway’s Game of Life. Game of Life is a fun little simulation to implement, it can be confusing at first to people that haven’t studied it before but if you know the problem well you can easily implement it in under an hour. However, the point is not to code Game of Life. Completing the challenge is not the aim. The goal is the experience and learnings from pairing with your peers and trying new things. In particular, practicing or learning (it was a first for most of our attendees!) TDD and pair programming.

The Structure

The day consists of five sessions of pairing with someone new each time, for about an hour each. In each session you spend 45 minutes programming and 15 minutes on a retrospective discussing what you learned or what you found interesting in that session, or simply showing off a particular implementation that you are proud of. After each session you delete your code (Because again, coding is not the point - the expderience is!). You may be surprised how many different ways the simple rules of Conway’s Game of Life can be implemented. And pairing with someone different each time enables you to get out of your own typical ways of coding and see how other people outside of your bubble look at a problem.

But if you get bored of doing the same problem repeatedly, you can switch it up with constraints. Constraints throw a wrench in your usual way of coding to see if you can learn something by doing things differently. Some of them are quite funny ones like not using an IDE or even writing all your code on paper. Others are interesting ways of working together such as ping-ponging, in which you switch which one person writes the tests then you switch and the other makes the test pass. There are also a lot of interesting software design constraints that I think one can really learn a lot from such as never using primitives, or only having one level of indentation, never using mutable variables.

The Day

The day went very smoothly. Quite a few people were new to TDD and pair programming so it was really interesting to see their experience and their reactions. It lead to a lot of interesting conversation as to their experiences and the value of these practices. It was really interesting to see people getting up to speed throughout the day, too: In the first two sessions a lot of people struggled to get one passing test, but by the end of the day most pairs were getting substantially far through the challenge in just 45 minutes and we had several pairs that managed to complete it.

For those struggling to get the hang of it I shared an example of a solution written in Rust that some friends and I had made and going through that during one of the retrospectives seemed to help. We also found that many people were finding it difficult to get started due to difficulty with setting up new projects, installing and configuring the right test framework or getting tests to run. This is something everyone got better at as the day went on but we also suggested Code Retreat’s starter repo as a faster way to get started, and that seemed to help some pairs.

We’ve also asked some of the attendees to share their experiences of the day. Robert Firek had this to say:

Not my first time, and probably not the last one. As usual, I enjoyed seeing how the same problem can be solved in so many different ways. Especially, I could share my experience. Great to see that so many people are still happy to meet and learn something new

Chris Eyre from Prima shared this:

I paired with 5 different people each with a varied level of experience. 4 of the attempts I used Elixir and one in Python. It’s a great way to work with a range of people that you would not normally meet. I would recommend this to anyone getting into TDD (or if you are already into TDD and want to share).

Magda Firek who started her programming journey earlier this year shared this:

It was the first workshop of this type that I participated in. It is also the first time I have dealt with the TDD technique, and I can say right away that I will use it in the future. Connecting with programmers who code in different languages was very interesting. I’ll join next year for sure to see how my knowledge improved. Massive “Thanks” to Codurance for organising.

The End

So, that was Global Day of Code Retreat 2022. If you think this sounds awesome, join us at Code Retreat 2023 next year! But you don’t have to wait until then, London Software Craftsmanship will be hosting lots of other regular events in-person in London before then.

Rust Macros - The What, Why and How

Thu, 26 May 2022 00:00:00 GMT

I delivered a “Thursdays Matter” talk with SkillsMatter about Rust macros; what they are, why to use them and how they work.

Introduction to Rust

Thu, 21 Apr 2022 00:00:00 GMT

At codebar festival 2022 I gave a talk introducing newcomers to the Rust programming language. It covers what Rust is, why you might want to use it and where to go to learn more.

Want to learn a new skill? Start a study group, here's how

Mon, 17 May 2021 00:00:00 GMT

Hoping to broaden my programming horizons in 2020 I studied F#, but I didn’t make as much progress as I’d hoped. My objective in 2021 is to pick up Rust, and I wanted to try something different this time. Thinking back to a study group I’d been part of in the past, and to group projects and such in my student days, I saw the following advantages to study groups:

Goals and deadlines - a group has to move to some cadence that is agreed up-front by all involved. Something like “Let’s read a chapter every week”. This gives you a specific, measurable goal and a timeframe to complete it in.
Motivation - I find that it is easy to put off something if you’re doing it for yourself. But if you’re doing it for someone else? In my experience that is much more motivating. Studying together involves making commitments to keep up with the reading pace and to prepare material for a study session, so you get that extrinsic motivation.
Feedback - once you’ve read and practised some particular aspect of what you’re trying to learn, how do you know you’ve understood it correctly? In a group setting you can share your understanding with one another and get feedback on whether you’ve understood the material correctly.
Fun - it is much more fun to learn along with friends and colleagues than to slog away by yourself.

So, based on that I started a study group in January. We meet every other week, regularly get 5-7 people in attendance, read The Rust Book and have fun while we’re at it. It has been great- I feel I’ve easily learned as much about Rust after a few months as I did about F# in all of 2020. So if you have something you want to learn- any kind of skill or knowledge- I suggest looking for others who are interested in it too, from your workplace and among your friends, and studying together.

How

Okay that was the “why”, here is the “how”. The first thing is to create a channel or group chat (in Slack, Teams, Discord, or whatever application). You’ll need this space to coordinate together. Then drop some messages about what you want to study and the name of the channel into several prominent, related channels/group chats. Don’t be shy, make sure as many see it as possible. In my case I dropped my message in several of the most populated channels in my work Slack but still had people only finding out about it and joining over a month after we’d started.

It is important to create your channel first, because it gives anyone that’s interested a place to register their interest and host discussions- otherwise you have several disjointed discussions in different channels. In my case I didn’t create the channel until later and exactly that happened.

Lastly, you want to big it up in meetings and water cooler chats with friends and colleagues too. Not everyone watches group chats like a hawk, but word of mouth may reach them.

A modestly successful first message

Then I’d suggest you invite anyone that showed an interest, give it a little time to see who joins, and start openly discussing how this group might work. In my case I made it clear from the start that I didn’t intend to dictate how this would work. I think that helped to get people engaged and invested in the group. I did this by asking for input from the group on every decision and taking votes (posting the options and voting with emoji reacts works well). There are a bunch of things you should decide together:

How often will we meet? I found every other week works well
What day & time will we meet? It should be a time that works for as many people as possible, but at the same time you can’t please everyone.
What will we study? There are probably numerous resources on your subject, and you need to agree on one shared goal.
What will the format be?

Format

That last one, the format, is important. You should agree on the format between yourselves and even iterate on it as need be in the future, but let me offer the format we’ve settled on as a template. I’m not sure of the exact origin of this format, but it’s based on something my colleague Alfredo suggested in a previous study group. The format goes like this, chronologically:

Greetings/chatting
Ask who’s completed the reading, find out where we all got to and how we found it.
For each section of reading we’ve completed since the last meeting someone makes a presentation
- This can take any form- slides, a live coding demo, a conversation. It’s worth discussing with your group which presentation styles work for them and keep that in mind, but there’s no need to be prescriptive.
- Everyone should feel free to interrupt and discuss. The presenter should put their understanding of the material up for scrutiny. The ensuing discussion challenges any misunderstandings that we may have unwittingly picked up and ensures everyone has the same shared understanding.
- Presentations can be interactive- I find that throwing in a Kahoot quiz works great.
Discuss any exercises we’ve completed since the last meeting- what was challenging or confusing, how did you find it?
- In our case we use the Rustlings code exercises and Exercism Rust track to practice between meetings. Each of the Rustling exercises are mapped to chapters in the book, so we can ensure they’re relevant to what we’re reading.
Decide how much of the material we’ll commit to reading before the next meeting
- The word “commit” here is key- this is something you’re all committing to completing within a timeframe, so don’t be overambitious. Ensure it is something that everyone is comfortable with.
- We look up the page numbers of the next few chapters so that we know how long they are, then compare that to how many pages we got through in previous iterations.
- If, based on the previous exercise, we don’t think we can complete the next whole chapter then we’ll split it into smaller sections.
  - Estimating and splitting work, informing planning by how much we completed in previous “sprints” - you may notice that this is similar to how agile planning works. I suppose the same principles apply everywhere.
Decide which exercises we’ll commit to completing before the next meeting
- As above, but with exercises (in our case, coding challenges. But it could be sheet music to play, or foreign language books to read, or whatever).
- Ideally these should be relevant to the material. In our case this was easy because the Rustling exercises map to the chapters in the book.
Ask for volunteers to present at the next meeting
- They will prepare a presentation of some sort to share their notes and thoughts on some section of the material for the next meeting.
- These make up the presentations (as we saw at the start of the format) for the next session.
- Split the material into sections however you like- we often split long chapters into two separate presentations.

Lastly, feel free to switch up the format on occasion. This keeps it fresh and less repetitive. In my case after we’d read the first 10 chapters of The Rust Book I recommended doing a coding dojo session, to apply our new skills practically with some pair-programming. The rest of the group was keen and the session was really fun and helpful, and switching up the format helped to keep the group from feeling repetitive or stale.

Conclusion

I really recommend this approach, it has worked great for me. If you try it, let me know how you get on. And if you know any other study group formats let me know, I’d love to try some new ones out.

Book Review- "You First; Inspire Your Team…" by Liane Davey

Thu, 30 Jan 2020 00:00:00 GMT

My reading material throughout most of December was You First by Liane Davey, a book about dysfunctional teams and how to not be part of their problem. Davey establishes the term “Toxic Team” to model teams where “the lack of alignment and the poor dynamic are a threat to both the productivity of the organization and the engagement and well-being of individual members”; in other words, teams whose members don’t get along and/or can’t work together frictionlessly to the point that it harms the team. I very much appreciate the “well-being of individual members” part; Davey approaches this problem not just as an obstacle to the organisation’s productivity, but to the quality of life and health of the inviduals in the team.

Davey models these toxic teams as having distinct types;

Crisis Junkies: a team that has to be constantly fighting fires, and otherwise isn’t productive.
Bobble Heads: a team that never disagrees with or criticises their leaders
Spectators: an apathetic team wherein no-one presents alternative ideas
Bleeding Back: a team where trust has broken down, but members act happy in public and take out their frustrations in back-channel, private discussions.
Royal Rumble: a team that can’t come to a consensus, wherein members commit viscious personal attacks against one another rather than having an objective debate.

The first half of the book details the model that Davey has established after years of experience business consulting, and during her PhD at University of Waterloo in Industrial/Organizational Psychology. The second half introduces positive behaviours one can use to reform a toxic team. One of the central promises of the book is that by exhibiting these positive behaviours just one person can turn a team around. This second half of the book I found to be much more interesting than the first, it’s very self-reflective; Davey ernestly encourages you to take a long hard look in the mirror and ask yourself what you can do better. It’s a kind of tough love that she espouses like so:

All I can do is hold up the mirror. You have to do the rest. If you really let the words sink in, some of them may be tough to hear… But if you’re willing to go there, you’ll find it’s liberating to understand why you are acting the way you are. If you’re behaving badly, it’s probably not because you want to. It’s more likely that you started using a coping mechanism to survive during a particularly hectic or vulnerable time. Taking a shortcut helped you to survive, but now you’re seeing the negative effects.

Insights

Amplifying Other Voices

One part of the book discusses “Amplifying Other Voices” and I felt it was very relevant to Software Engineering. Davey discusses a team in which every member was a blue-sky thinking, big ideas person except for one. This lone, process-minded pragmatist found themselves ostracised from the team and had to work hard to make their point of view heard.

Unfortunately this reminds me of how software teams tend to consist mostly of developers, with one or two attaches representing other interests; a Quality Analyst, a Product Owner, maybe a Business Analyst. I’ve often found that a tight-knit culture emerges around the developers with their shared interests (technical excellence, ways of working, prioritising technical debt) while non-developers may have other motivations (Timely delivery, pressure from upper management, user experience). If your team has this problem particularly badly you may catch yourself referring to people in these roles as “The business”; as in “We should see what The Business thinks of this; let’s ask Dave”, as if the developers weren’t part of the business at all.

Davey suggests that we can solve these problems by amplifying other voices. Not that we need be guilty about the blitheness that occurs when you get a bunch of computer nerds in the same room, but that we should be aware of how a boisterous group may inadvertently drown out the minority, that our perspective isn’t the only valid one and that to achieve a measure of balance we need to amplify the minority’s voice before we can have a diverse discussion that appreciates every angle.

Start with a Positive Assumption

I’m a bit of a cynic. I don’t trust people easily, and I’ll often second-guess what another person’s motivations may be. So this is the chapter that spoke to me the most. In it Davey suggests a simple principle to live by; start with a positive assumption. Anyone you have dealings with, assume that what they say is in ernest, that they’re committed to their work and have no ill intentions.

Davey describes four areas of trust: connection, competence, reliability, and integrity. In this context, I’m a little confused as to what “connection” means; but Davey uses it to say that you should assume everyone you meet is “a decent human being who got up in the morning and came to work to do a good job and make a contribution”. The other three are self-explanatory, but I particularly like Davey’s reasoning for why we should trust one another’s integrity:

If you sit across the table from someone assuming that they are going to screw you, you are actually giving them a reason to do it.

Davey suggests that we can change our behaviour by stifling the impulse to make a cynical assumption, consider the context that this person sees the topic from, get more information by asking open-ended questions, and listen for clues: try to understand the underlying emotions and beliefs that may have informed the other person’s perspective, and look for any evidence they may have that contradicts your own view. This method reminds me very much of critical thinking; go out of your way to expose yourself to contradicting views, and ernestly analyse their merit. Don’t trust your own assumptions, but rather accept your falibility and practice self-correction.

Since reading You First this phrase, “Start with a Positive Assumption”, has been rattling around in my head. Whenever I find myself questioning someone’s intentions, I remind myself to start with a positive assumption. By coincidence when I was attending the Codurance OpenSpace recently one of my colleagues initiated a round-table discussion on the topic “Assume Good Intentions”, which seems to be the same concept. They described how they’d been practicing it as a mantra of sorts for years until they no longer thought of it at all, and such trust came naturally to them. I hope to reach that point one day, too.

Overtime

I don’t agree with Davey on everything. In particular, we differ on our view of overtime. I feel strongly that overtime should not exist in business. Where overtime is seen it is a symptom that something is wrong (Understaffing, bad deadlines, etc), and by taking part in overtime you mask the problem rather than solve it. You accept being overworked, and the struggling machine continues to maintain it’s mirage of an orderly contraption. Better, I think, to let the machine break and in doing so expose the problems inherent with it.

I often find my views clash with a pervasive business culture wherein overtime is viewed as a good thing that should be praised. Several times in the book Davey espouses this view, like so:

Sometimes the inner circles forms among a group of people who work longer hours and go the extra mile. These people have learned to count on one another to get things done when it really matters. Next time a project requires extra hours, roll up your sleeves and find a way to be useful.

In this case, I would say the “inner circle” is the problem. By letting work take over their lives they’re not only sacraficing their own work-life separation but, as Davey illustrates here, harming other people’s quality of life by creating a growing culture of overtime.

Conclusion

I’ve certainly been on teams that would fit one or more of the book’s definitions of toxic teams, and you probably have too. Perhaps the members of these teams didn’t disagree as violently as the two argumentative foxes in the banner above, but nonetheless there were problems.

I’ve worked in teams where contentious issues were discussed in secret, where everyone is managing a list of secrets. I’ve worked in teams where colleagues form into cliques that oppose one another. I’ve worked in teams full of apathetic spectators, because management has taught them time and time again that trying to drive positive change is fruitless. Knowing how to handle problems like these effectively is surely part of the professional mastery we should all strive for. So the next time I find myself in a toxic team I’ll reach for Davey’s guide and put it into practice.

Testing Your Website for Visual Regressions With BackstopJS

Sat, 25 Jan 2020 00:00:00 GMT

First posted on the Codurance blog

Snapshot testing compares a “snapshot” of the output of a prior version of your software to output from the latest version, to check for unintended changes. When a difference is found you either approve it, by updating the expected output snapshot, or fix the cause of the difference.

Visual regression testing is a form of snapshot testing that tests a web front-end. It goes beyond testing the markup or layout by testing the rendered page captured in an emulated browser. As such they can “catch CSS Curve Balls” as BackstopJS says.

BackstopJS is a framework for visual regression testing, written in Javascript. It treats your web service as a black box, so your website doesn’t need to be written in Javascript to work with BackstopJS. One of the benefits it offers is a very comprehensive and helpful diff between your snapshots, embedded in a HTML report that it generates. An example below shows how the scrubber diff method allows you to see both test and reference snapshots simultaneously. You can move the red line to change where the boundry lies.

This article will explain how to set up BackstopJS and some good practices I’ve picked up from using it in anger. You’ll need to have some awareness of Docker Compose and Yarn or NPM.

Why

Why use visual regression testing? I’ll assume that you appreciate why testing as a general practice is necessary, so here are several scenarios that visual regression tests will catch and other testing techniques won’t:

CSS regressions: Often we’ll make a change to a CSS style to move that one button into the space we’d like it to be, but how do we know that it hasn’t had an undesirable knock-on effect on some other part of the website that uses the same CSS class? Instead of checking every element that may match your CSS selector manually (which we will most often forget to do), run your visual regression tests.
Responsiveness: Most often we’re working on a 1080p screen, but many (perhaps most) of our users will be using their smartphones. Visual regression tests can test an assortment of different screen sizes, giving you confidence that your changes haven’t broken the responsive page at other sizes and saving you time manually testing the page at different resolutions.
Dependabot: It’s a wonderful tool that saves you from manually keeping all of your dependencies up-to-date. In my current team we use dependabot aggressively; we have extensive testing and auto-merge any Dependabot PRs that pass all our tests. In fact, in terms of PRs or commits it’s the most active member of our team by far. However, you can’t place your trust in Dependabot’s auto-merge in the front-end if you aren’t testing for visual regressions. Before we introduced visual regression testing we had instances where Dependabot would automatically update to a new version of a dependency that introduced a visual change (at one point even removing a prominent navigation link), and it would be automatically merged and deployed to production. We trust Dependabot to change our software’s behaviour because we have extensive tests of that behaviour, and we trust it to change our software’s visual appearance because we have visual regression tests.

How

Setting up BackstopJS isn’t difficult. But I’m going to make it more difficult because we want to set it up in a way that’s repeatable (ie it always has the same result for the same version of the software) and automated. You don’t want your visual regression tests to be “flaky”, to pass some times and not others. It’s worth putting extra effort in to get this right, otherwise they may be less than useful. And you want them to be automated because that way you can trust yourself and others not to forget to run the test or update the snapshots, and you can use them as checks for Dependabot’s auto-merge.

Ultimately, your workflow should look like:

You push some horribly misguided code, introducing an unintended visual change.
Your CI runs the visual regression tests and they fail, turning your build red.
You look at BackstopJS’s test report from your CI, and spot the problem.
You decide whether this change is acceptable, and either update the snapshots to this change or fix the code that caused this change.

Docker

The first step to creating a repeatable visual regression test is to run the tests on the same platform every time. Otherwise, you’re in for trouble. Small changes in things like font rendering between operating systems can prevent the reference snapshots generated on your local machine from matching the test snapshots generated on your CI server. And if your CI server has multiple test runners on different platforms you’ve got even more unpredictability on your hands. To get around issues like these we use Docker containers via Docker Compose. This guarantees the same platform for every test run. This approach also has the advantage that you don’t need to install BackstopJS locally and end up with different versions of it on each developer’s machine; instead you have one consistent version, and Dependabot can keep it up-to-date. The disadvantage is of course that it’s slower.

Add the following entry to your root docker-compose.yml (create one if necessary):

  visual_regression_tests:
    image: backstopjs/backstopjs:4.4.2
    volumes:
      - ./test/visual:/src

This describes a visual_regression_tests Docker container using the official backstopjs image, version 4.4.2. The version can be left out, but it’s important that it be there for repeatability. You can use Dependabot to keep it up-to-date by creating a Dockerfile for it (until Dependabot adds Docker Compose support), which is described below. Of course, you should copy the latest version number from the BackstopJS Docker image releases and use that; 4.4.2 may be outdated at time of reading. If you’re using a Docker container for your website as well you should add a depends_on entry to that container.

The last part is the key; the volume configuration ./test/visual:/src. This maps the local relative path ./test/visual to /src in the container. You may change ./test/visual to any relative path you like, but /src must be constant because that is where BackstopJS will look inside the container for it’s configuration.

BackstopJS

Previously, I said that with Docker developers don’t need to install a local instance of BackstopJS on their machines. This is true, but with one exception: You. That’s because for our next step we’re going to create the BackstopJS configuration, and you’ll need to use Backstop to create a default instance of the configuration. So try this:

yarn global add backstopjs
mkdir ./test/visual
cd ./test/visual
backstop init

First we install BackstopJS (NPM alternative: npm install -g backstopjs) and then create the folder where our container expects to find the configuration (so change this as you like, but ensure it’s consistent with the Docker Compose volume). Then we open the folder and initialise a BackstopJS config there. This creates a few files; backstop.json, and backstop_data/engine_scripts. The engine scripts are basic defaults that determine how to run the browser emulator. Unless you’re doing something unusual you shouldn’t need to change most of them.

Before going any further, create a .gitignore in your BackstopJS folder with the following entries:

backstop_data/bitmaps_test
backstop_data/html_report

This will ensure that the test snapshots and HTML reports generated by Backstop are ignored by Git. You don’t want to commit these to version control, but you do want to commit the other folder it generates; the snapshot references to test against.

The backstop.json file is your main means of interacting with BackstopJS and to start with should look something like this:

{
  "id": "backstop_default",
  "viewports": [
    {
      "label": "phone",
      "width": 320,
      "height": 480
    }
    …
  ],
  "onBeforeScript": "puppet/onBefore.js",
  "onReadyScript": "puppet/onReady.js",
  "scenarios": [
    {
      "label": "BackstopJS Homepage",
      "cookiePath": "backstop_data/engine_scripts/cookies.json",
      "url": "https://garris.github.io/BackstopJS/",
      "referenceUrl": "",
      "readyEvent": "",
      "readySelector": "",
      "delay": 0,
      "hideSelectors": [],
      "removeSelectors": [],
      "hoverSelector": "",
      "clickSelector": "",
      "postInteractionWait": 0,
      "selectors": [],
      "selectorExpansion": true,
      "expect": 0,
      "misMatchThreshold" : 0.1,
      "requireSameDimensions": true
    }
  ],
  "paths": {
    "bitmaps_reference": "backstop_data/bitmaps_reference",
    "bitmaps_test": "backstop_data/bitmaps_test",
    "engine_scripts": "backstop_data/engine_scripts",
    "html_report": "backstop_data/html_report",
    "ci_report": "backstop_data/ci_report"
  },
  …
}

The first thing I’d advise changing is the viewports property. This property determines the resolutions that the site will be tested at. The default is not very extensive, and in my current team we’ve settled on the following viewport configuration:

  "viewports": [
    {
      "label": "small",
      "width": 640,
      "height": 480
    },
    {
      "label": "medium",
      "width": 814,
      "height": 768
    },
    {
      "label": "large",
      "width": 1066,
      "height": 814
    },
    {
      "label": "xlarge",
      "width": 1400,
      "height": 1050
    },
    {
      "label": "xxlarge",
      "width": 1600,
      "height": 1200
    }
  ]

The next interesting property is scenarios. A scenario defines a test, and you’ll want to add one for each major section of your website. With a blog for example you may want to test the blog page and the blog list page, so you would have two scenarios.

The real trick here that will lead you to either jubilation or despair is figuring out when to take the snapshot. Browsers, Javascript, web services and HTTP are all such fickle beasts; they may load slightly faster or slower each time you create a snapshot. For your visual regression tests to be repeatable you need them to create the snapshot only when the page has finished loading. If you don’t you’ll find many test failures caused because the font hadn’t loaded in yet, or a pop-up hasn’t appeared yet, or a HTTP request to an AJAX dependency hadn’t completed yet, et cetra. As such a lot of the scenario configuration options are about when to decide that the page has finished loading. This is the real meat of the configuration and each possible option is documented in BackstopJS’s readme, but a few key ones to highlight are:

cookiePath: This enables you to enter faked cookies into the browser emulator, this can be useful to send a token to an authenticated web service. Just set it to a relative path to a JSON file; the expected format is described in a sample file, engine_scripts/cookies.json.
url: This is the full address of the web page being tested. If you’re using a Docker container to host your site you may use the name of the container, like http://website:8080/myPage. Otherwise, you may run it locally and use something like http://localhost:8080/myPage.
readyEvent: Listen out for a console log telling you the page is fully loaded before starting. This is useful for repeatability.
readySelector: Similar to the above, this configures Backstop to wait until a particular element (defined by CSS selector) is appearing before starting. I recommend using this setting and setting it to something that won’t appear on any of your error pages. If your service doesn’t work during a visual regression test you may not know until after you get the report and are staring at a diff between your reference and a 404 page. But if your readySelector fails you get a timeout error in the output that lets you know that the expected page hasn’t loaded, so you get the feedback sooner.
delay: Avoid using this setting if you can. It allows you to set an arbitrary time to wait for the page to load before assuming it’ll be ready to test. By default it is 0, which means no delay. If you find yourself using this setting, it’s because you haven’t found a reliable method to tell the browser that the page is loaded. You should only use this as an absolute last resort.
hideSelectors/removeSelectors: If you have some problematic element on the page that you either can’t rely on to load in a reliable, timely fashion, or which has some random element that changes each time it’s loaded then you can hide those elements using either of these properties.
scrollToSelector: BackstopJS will capture the entire document, not just the visible section (unless configured otherwise). However, you may want to trigger some event via scrolling. This setting makes Backstop scroll to a particular selector.
selectors: By default BackstopJS will capture the entire document. But if you want to test a specific set of regions, you can use this to limit the elements used to generate the snapshot. It’s the opposite of hideSelectors/removeSelectors (but they can be used together). It’s especially useful when you want to break a complex page down into smaller parts; you’ll get more specific feedback on individual components, making regressions easier to identify.
misMatchThreshold: The degree to which two snapshots must be different before the scenario fails. This defaults to 0.1 (That’s 0.1%, not 10%) and I wouldn’t increase it without good reason.

Running the tests

At this point you should be able to run your tests. From the root of the project do docker-compse run visual_regression_tests reference; this will generate your first reference images. Then try docker-compose run visual_regression_tests test; this will generate new references and test them against the last ones you captured. I suggest recording these commands as scripts, so that every developer doesn’t have to remember them. In Yarn/NPM we add a script to package.json for run these commands, otherwise we create a shell script inside a ./scripts folder.

Don’t worry if your tests aren’t passing the first time; I’ll explain some ways that you can improve their consistency and reliabiliy.

Dependencies

Just in case I haven’t said this enough times: Repeatability is key. One of the obstacles to this repeatability is ensuring that your dependencies are consistent. If you depend on a web service or a database that sends you some data to display on the page, then that service needs to send the same data every time the visual regression tests are run. This means that you need the capability to fake your dependencies. If you depend on a database, then you may want to achieve this by creating a Docker container of your particular database dependency with some minimal fake data. If you’re dependent on web services, then I’d recommend using Mockingjay Server. It’s a Docker container around the mocking service Mockingjay. This makes for a painless and platform agnostic way to fake your dependencies with web services that respond with fake data. Just add something like the following to your docker-compose.yml:

  fake_my_service:
    image: quii/mockingjay-server:1.10.4
    volumes:
      - ./test/fakes:/fakes
    command: -config=/fakes/my_service.yaml -port=9099
    ports:
      - "9099:9099"

For this to work you have to have a directory, here specified as ./test/fakes, with a YML file that specifies the endpoints to fake following Mockingjay-Server’s format. This may include multiple fakes for different services. We specify which fake file to use in the command property. Then, we just configure our web service to talk to this fake service when the tests are run. This way we know that our service will reliably, repeatably generate the same output and that’s a huge benefit to our testing.

Custom Fonts

Downloading a non-default font from some server somewhere will take an unpredictable amount of time, so it harms our repeatability. Rather than reaching for that unreliable delay setting, however, we can pre-install the fonts on the Docker image to get around this problem altogether. Simply create a Dockerfile inside your BackstopJS directory with an entry like the following:

FROM backstopjs/backstopjs:4.4.2
RUN apt-get update && apt-get install -y fonts-lato
RUN apt-get update && apt-get install -y fonts-font-awesome

This is a very basic Dockerfile that extends the official BackstopJS image (remember to include the version!) and uses apt-get to install the requisite fonts. This way the browser emulator won’t need to download the fonts as they’re already installed. You should be able to find the package name of any font you need by searching Debian’s package registry. Then you just need to change your docker-compose entry to build your BackstopJS directory, like so:

  visual_regression_tests:
    image: build: ./tests/visual
    volumes:
      - ./test/visual:/src

Continuous Integration

You’ll want to add a step to your build to run the visual regression tests. If you created a script to run the tests earlier then you can simply plug it in here. Because we’re using Docker you needn’t install BackstopJS on your CI server, and it will play nicely with Docker-based CI systems like CircleCI. There is an important extra step though; you need to extract the build artifact from BackstopJS. Otherwise, you won’t be able to see why your tests have failed.

For Jenkins you can achieve this with the HTML Publisher Plugin. Though the official jenkins support guide involves setting up a Jenkins job in the traditional web interface, I’d advise against that and use the declarative Jenkinsfile method. If you already have a declarative Jenkinsfile just add something like the following to your always post-step in your Jenkinsfile:

post {
  always {
    publishHTML(target: [
      allowMissing: false,
      alwaysLinkToLastBuild: true,
      keepAll: true,
      reportDir: './test/visual/backstop_data',
      reportFiles: 'html_report/index.html',
      reportName: 'Visual Regression Tests Report'
    ])
  }
}

Testing Repeatability

Lastly, to ensure repeatability you can actually repeat the tests. When first setting up any tests involving browser emulation or browser automation I won’t accept a passing test as correct unless it passes many times, so I can be confident that it isn’t going to prove unreliable (“flaky”) in the future. A simple Bash script like the following will suffice:

set -e #ensures the script will halt if any of the test runs fail

for i in {0..19}; do
  yarn test:visual
done

Sure, it may take a while to run. Just leave it running in the background while you’re doing something else.

Conclusion

There are a number of other approaches to running BackstopJS, but this is the best type of setup I’ve found. Hopefully I’ve equipped you with everything you’ll need to get up and running with repeatable BackstopJS visual regression tests on your websites. Let me know how you get on.

Open Rights Group Conference 2019 Review

Sat, 02 Nov 2019 00:00:00 GMT

In July I attended this year’s Open Rights Group Conference, a one-day multi-track conference organised by the Open Rights Group (ORG). ORG is the UK’s biggest advocacy group for digital rights and campaigns on issues like surveillance, net neutrality and censorship. Despite being a member for many years this is only the second time I’ve attended an ORGCon, and it was honestly one of the best tech-related events I’ve been to all year. It’s not exclusively for techies, on the contrary I was in the minority. There were many more journalists, activists and lawyers there. I’ll dive into what made it great, describing each fascinating session I attended.

Session one- Keynote by Edward Snowden

Image from @greekemmy

For the first session of the day American whistleblower Edward Snowden gave a keynote speech followed by Q&A, via videocall from Russia. No-one in the room missed the irony of relying on Google’s Hangouts service to talk at a digital rights conference, and the audio was appropriately unreliable. In spite of that Snowden delivered an enlightening, inspiring speech. On discussing his whistleblowing, he made an interesting clarification: That his motivation was about democracy, not about surveillance. What disturbed him more was not the orwellian surveillance of hundreds of millions of innocent people, but the fact that none of them had agreed to it. The fact that there’d been no debate, no chance to object before this policy was implemented. Other highlights from the session:

He very heavily criticised the Ghost Proposal, a GCHQ bid to encrypt all our communications with a key that only the government can decrypt¹. Specifically he referred to how the NSA lost track of some of their most valuable digital assets that were then used by malicious actors cause huge amounts of damage², and questioned whether the “rinky-dink UK government” (with tongue firmly in cheek, I’m sure) would be able to protect a set of keys that anyone could use to decrypt millions of people’s communications.
“The law does not protect society, society protects the law”; Snowden made the argument that laws are tools and only effective so long as conscientious citizens use them and uphold them.
- He further arguded that technology is another tool that, like law, can and should be utilised to enforce human rights. One obvious example is encryption; it should surely be illegal to eavesdrop your communications without due process, but encrypting one’s communications is another way to enforce that principle.
He ended his speech on an inspiring note; “We will lose this year and the next one, but we will win- it’s a way of life, a system of beliefs that will not go away, what we did to resist governmental abuse of surveillance will matter to our children” (paraphrased to the best of my memory).

Session two- “How Can I Trust What I See Online?” panel from Teen AI

Image from Acorn Aspirations

This panel consisted of four teenagers from Teen AI who’ve been studying computers, security and AI as well as attending various tech events for young people. It was really inspiring to see young people so interested in technology, they were each well in advance of where I’d been at their age. Unfortunately I’ve forgotten exactly which panelists said what, having misplaced some notes. Nontheless, my highlights were:

One panelist made the point that deepfakes may actually make us trust what we see less, thereby making us more skeptical. This may be a positive change in society rather than the bleak forecasts that we usually associate with deepfake tech.
One panelist when aksed whether young people need to be taught how to be safe online responded “No, old people do”. They went on to discuss how young people are that much more tech-savvy and more likely to understand what they’re looking at, while it may be that older people are more susceptible to misleading or malicious content.
“If we don’t understand technology we will be controlled by people who do”- a wonderful response from one of the panelists discussing their motivations for learning about technology.
Another panelist said something that was very novel to me, when making the point that technology is not inherently good or bad. I’ve always thought of facial recognition as being an inherently orwellian and oppressive tool, but they cited a much more positive example: Police in New Delhi are using it to locate missing children³. This is one of those wonderfully simple yet effective ideas; go around taking pictures of children at orphenages, use facial recognition to compare them to the faces of missing children that their family have posted online. In just the first four days they identified 3,000 missing children.

Session three- Can Tech be Truly Ethical?

Image from David Raho

This was a fantastic session, it really began to open my eyes to how complex ethics in technology can get. We had Paul Dourish (University of California), Lilian Edwards (Newcastle University), Ann Light (University of Sussex) and Gina Neff (University of Oxford) discussing the topic, and each demonstrated obvious expertise in the subject. Among numerous enlightening moments were:

“Ethics washing”- a new term to me; it refers to the tendency of organisations to create an ethics board, often entirely internally with no public oversight or transparency, and declare themselves to now be ethical. The accusation is that such organisations are trying to escape true scrutiny and regulation by saying “We’re ethical now, so there’s no need to regulate us”.
Ethical behaviour can be difficult to enforce because of combinatorial complexity. This is an especially significant issue with Internet of Things devices, filling homes and interacting with one another. You may ensure your software acts ethically in as many ways as you can think of, but your software will very frequently find itself in situations that you hadn’t imagined. When a thousand different devices are capable of interacting with one another, we can’t predict every interaction ahead of time.
- This may be a good use case apply property-based testing.
One person in the audience was a member of a worker-owned software business, which is the first time I’ve heard of such a thing. It sounds like a great development.
“How do we stop a raging bull”- while we’re clamouring to figure out what technology’s role in society is and how it affects all of us, technology continues to develop rapidly and crosses new boundaries into new parts of our lives every day. It’s a lot easier to start a fire than to put it out; how can ethicists and human rights campaigners keep from always being caught on the back foot?
- It was suggested that we should look at business models, and the way different revenue sources can affect a business’ motivations. Right now, huge swathes of the tech industry are profiting from automated surveillance of their users. What revenue sources could offer more ethical motivations?
It’s one rule for them, one rule for us: San Francisco has banned facial recognition within the city⁴. Yet many of the city’s occupants are affluent technologists making their money by continually improving the very same problematic technologies that they’re exempt from, and much of the rest of the world is increasingly subjected to.
Do Artifacts Have Politics was dropped in as a piece of recommended reading.

Session four- Facial Recognition is Now a Reality in the UK

This panel was packed with talented and interesting people- a senior technology correspondent at The Financial Times, the director of Big Brother Watch, the Chair of the London Policing Ethics Panel, and Sian Berry co-leader of the Green Party. The topic was on facial recognition in the UK, and in particular how it’s been deployed at Notting Hill Carnival in recent years. The Met Police started using the technology without notice or discussion, and studies have found that it’s grossly ineffective⁵. My notes from the session:

The Met Police didn’t admint to using the technology until Big Brother Watch pushed the issue.
There is no basis in law for the use of facial recognition. No-one knows whether it is something that needs to be consented to under UK law, or what limitations there are on it.
There is an ongoing legal battle, brought by Big Brother Watch, to decide the legitimacy of facial recognition in UK law⁶.
Suzanne Shale was a great balancing voice representing the Policing Ethics Panel, and the discussion was much less partisan or one-sided for having her there.
Sian Berry assured the audience that progress was being made, and the Met Police would be putting a memorandum on place on the use of facial recognition technology.
Another issue was raised regarding regulation of private facial recognition, taking the cameras in self checkout machines as a particular example. They’re constantly facing outward from behind the screen and have a clear view of the consumer. They could be, and may currently be, used to build facial recognition databases and map them to our cards and purchasing histories. There’s currently little if any regulation to prevent this.

Session five- The Encryption Dilemma

This was perhaps my favourite session, it was really remarkable and speaks to the integrity of the organisers and the whole community. The first thing to notice in the picture is that Dr. Ian Levy is missing a photograph. This is because he was added at the last possible minute. This session was supposed to be a one-on-one interview, where Cori Crider and the audience asks questions of Nate Cardozo, privacy policy manager at Facebook and former senior policy-maker at the Electronic Frontier Foundation (EFF), the biggest digital rights advocacy group in the United States.

Today Nate mostly works with WhatsApp, which is setting new standards for private digital communications by popularising the end-to-end encryption model. This model is infuriating to the likes of GCHQ because it removes their ability to intercept communications. GCHQ’s Ghost Proposal is supposed to be a middle-ground where end-to-end encryption can exist between a server and their client, but the state can sit in the middle and read everything. Nate is a big time digital privacy advocate and a very outspoken critic of the Ghost Proposal. He even wrote a well publicised article for the EFF in which he called the Ghost Proposal a “backdoor by another name”⁷.

Enter Dr. Ian Levy, co-author of The Ghost Proposal and Technical Director of the Cyber Security Centre (a part of GCHQ). I gather that he found out about this interview and argued that that, as it was obviously going to center around the ghost proposal, he ought to have an opportunity to defend the proposal. So at the twilight hour the planned session turns from an interview into a debate, and Ian is added to the session. I think it’s really wonderful that we can have a nonpartisan debate like this. I really admire Ian for coming to the event knowing that the audience would largely disagree with him and to ORG, Cori and Nate for doing the right thing by giving Ian this chance, and avoiding the conference becoming an echo chamber. Here are my highlights:

Ian went to great lengths to make the point that he wishes to genuinely engage with the digital rights community, to come to a common understanding and find some resolution for the encryption debate that satisfies everyone. Needless to say this is something that GCHQ and the like have an extremely poor track record of, so it’s great to see someone making a change. “The fact I’m here should tell you something”, Ian says, and it does.
Ian insists that the Ghost Proposal is nothing more than a proposal and says that he wants to use it to trigger debate, to iterate on this idea and get public discussions moving.
The end-to-end encryption model is coming to Facebook Messenger too.
Questioned on WhatsApp security, Nate assures the audience and adds: “If I leave in a huff, stop using WhatsApp”.
Ian responded really well to a difficult question- “If we introduce a backdoor, won’t terrorists just write their own encryption?”. Ian responds that he’d love that. Cryptography, he argues, is hard. Unless you’re an expert cryptographer, when writing your own encrypted communication software you’re almost certain to make a mistake that intelligence services can exploit.
Ian, perhaps as part of trying to humanise his work and reach out to people, recommends the “Top Secret” exhibition at the Science Museum as a great way to learn the history of GCHQ.
There was a great question from the crowd criticising the Ghost Protocol- “Are you leading the way by using it in all internal GCHQ messaging? Do the royals use it?”. Going back to the point of “one rule for us, another for them”; GCHQ would obviously resist any attempt to weaken their own communication privacy, as would the more affluent members of society.
The discussion also went briefly into Cloudflre’s DNS over HTTPs. Ian and GCHQ are very much not fond of this as it once again makes tracking people on the internet more difficult. Conversely, I must say I’m quite enamored with the idea.

Session six- Jack Poulson on Censorship

Jack Poulson is known for quite publicly leaving Google, where he was a Senior Research Scientist, over their “Dragonfly” project to censor search results in China⁸. The project would have blocked results for searches like “human rights”, “Nobel Prize” and “student protest”⁹. My notes:

Challenged on what he had expected his resignation to do, Jack pointed out the successes Google staff have had in changing their employer’s policy¹⁰.
Jack made a similar stance when he worked in academia, refusing to support closed access journals. He was threatened with tenure removal if he didn’t comply, and resigned two days later.
Google first shut down Google.cn because China attacked them, compromised their source code and more in order to track human rights activists¹¹.
Jack suggests that we too often discuss ethics in a future tense, as if it were theoretical- “Oh, what if technology were to harm society…” but tech already is harming society. It’s better, he argues, to refer to actual historical human rights abuses, such as when Cisco wrote software for the Chinese government knowing that it would be used to detain and torture dissidents¹².

Art installations

Lastly, there were a number of interesting art installations around the conference. I particularly liked these:

This display demonstrated the amount of land that Mark Zuckerberg has purchased, at great expense, around his home to ensure privacy. The author asks; are we entering a world where only the rich can afford privacy?

A children’s connect-the-dot game in which each dot is a subsidiary of Alphabet, Google’s parent company. A great way to visually demonstrate the sheer scale technology corporations are reaching.

Each talk in the biggest room was accompanied by an artist drawing a wonderful piece to summarise the topics covered in the talk.

Conclusion

ORGCon is great. I never imagined I’d learn so much about ethics in technology. There were some really unexpected and interesting sessions. I definitely recommend going next year, and perhaps you’ll consider joining the Open Rights Group.

Alex Hearn, The Guardian, Apple and WhatsApp Condem GCHQ Plans to Eavesdrop on Encrypted Chats ↩
Nicole Perlroth, David E. Sanger, New York Times, How Chinese Spies Got the N.S.A.’s Hacking Tools, and Used Them for Attacks ↩
Anthony Cuthberston, The Independent, Indian Police Trace 3,000 Missing Children In Just Four Days Using Facial Recognition Technology ↩
Dave Lee, BBC News, San Francisco is first US city to ban facial recognition ↩
Matt Burgess, Wired.co.uk, The Met Police’s facial recognition tests are fatally flawed ↩
Big Brother Watch, Big Brother Watch Begins Landmark Legal Challenge to Police Use of Facial Recognition Surveillance ↩
Nate Cardozo, EFF, Give Up the Ghost: A Backdoor by Another Name ↩
Ryan Gallagher, The Intercept, SENIOR GOOGLE SCIENTIST RESIGNS OVER “FORFEITURE OF OUR VALUES” IN CHINA ↩
Anthony Cuthberston, The Independent, GOOGLE EMPLOYEES SUSPECT IT IS STILL WORKING ON SECRET CHINESE ‘DRAGONFLY’ PROJECT ↩
Daisuke Wakabayashi, Scott Shane, New York Times, Google Will Not Renew Pentagon Contract That Upset Employees ↩
Elinor Mills, CNet, Behin the China attacks on Google ↩
Stephen Lawson, CIO, EFF says Cisco shouldn’t get off the hook for torture in China ↩

Matt.si

Deterministic(ish) Machine Configuration with Python

The code

Automatically registering GitHub SSH keys

Conclusion

'Rust 101: Understand the Hype (Devoxx UK)'

Large Language Models Are Drunk at the Wheel

How did we get here

Where are we

Infinite Possibilities, Uncontrollable Chaos

The craze

There but for a use case

Valid use cases

We’ve been here before

Why does this matter

Conclusion

Footnotes

Migrating AngularJS to Angular with a hybrid application

Why not (some other UI framework)?

Resources

Angular and AngularJS side-by-side

ng-upgrade

The Shell Strategy

Routing

Route syncing

Upgrading and downgrading services

Training

Courses

Knowledge sharing

Finding help

Conclusion

Introduction to Test Driven Development (TDD)

Rust Screenkatas

Rust Nation 2023 Review

Day One - Delivering a Foundational Workshop

Day Two, Session One - Grand Opening

Day Two, Session Two - TockOS Tutorial

Day Two, Session Three - Meeting the Game Developers

Day Two, Session Four - Moving Past Proof of Concept

Day Two, Session Five - SurrealDB

Day Two, Session Six - Rust Foundation Q&A

Day Two, Session Seven - Living with Rust Long-Term

Conclusion

ChatGPT Can Do Astonishing Things. But Is It a Helpful Development Tool?

What is ChatGPT?

Can it code

But what about tests

But what about software design

Other interesting uses

A Smart Search Engine

Translate this error message, please

Convert code from one language to another

Querying datasets

Limitations

Don’t Believe It’s Lies

Internet Not Included

How do I shoot myself in the foot?

Does it really save time?

Conclusion

Retrospective - London Global Day of Code Retreat 2022

The Code

The Structure

The Day

The End

Rust Macros - The What, Why and How

Introduction to Rust

Want to learn a new skill? Start a study group, here's how

How

Format

Conclusion

Book Review- "You First; Inspire Your Team…" by Liane Davey

Insights

Amplifying Other Voices

Start with a Positive Assumption

Overtime

Conclusion

Testing Your Website for Visual Regressions With BackstopJS

Why

How

Docker