Can one mistake truly be forgiven, especially if its impact spans decades and influences an entire industry?
When you search for "the billion-dollar mistake" on Google, most of the results point towards Tony Hoare and his creation of the null reference in 1965. In a presentation, he dubbed this concept "the billion-dollar mistake," a term that underscores the significant and costly impact null references have on the software industry.
Interestingly, for tech giants such as Facebook, Google, Amazon, and Apple, whose valuations range from $500 billion to $1 trillion, any programming error leading to a mere 0.2% drop in their valuation instantly becomes a billion-dollar mistake. This fact shows the enormous risks involved in software development for these corporations.
Origins of the null reference concept
Tony Hoare, popular for inventing the Quicksort algorithm and the ALGOL programming language, ironically gains more attention for his self-proclaimed "moment of laziness" than for these achievements. This blog will focus on the null reference - defining it, discussing the costly issues it can cause, and examining whether the industry has learned from Hoare's 2009 confession of his mistake. Despite the years that have passed, the question remains: As of 2024, do null references still result in headaches and billion-dollar losses?
In 2009, Tony Hoare shared a candid reflection on his invention, stating:
"I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object-oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years."
So, in 1965, while designing the ALGOL W programming language, Tony Hoare introduced the null reference to allow references to point to "nothing." As he said, this was easy to implement at the time, but later, it resulted in the common issue of null reference exceptions, which caused crashes and bugs in software.
What does NULL signify?
To grasp the concept of null reference effectively, it's important first to understand what NULL represents.
The simplest distinction is between NULL and the number 0, a comparison that often leads to confusion. Unlike 0, which indicates that a variable holds a value precisely equal to zero, NULL means the variable is without any value whatsoever.
For a better understanding of the mentioned terms, we can use the picture below (the picture includes non-zero and undefined values for better visualization).
Explanation of the difference between non-zero, 0, NULL, and undefined values
Having clarified the nature of a NULL value, we can now go into the concept of a null reference.
What is the null reference?
Essentially, any reference should allow the program indirect access to a specific piece of data, typically the value of a variable. However, a null reference is unique because it signifies that the reference points to an invalid or nonexistent part of the memory—meaning it does not point to any valid memory location.
This definition highlights the core issue: a null reference attempts to access data from an invalid or nonexistent memory location, leading to a null reference exception and, by extension, significant frustration for programmers.
Why are null values necessary for programming?
In software development, null values are a foundational concept integral to understanding and managing data across various programming languages. The introduction of null values, null references, and null pointers addressed specific, critical programming needs, enhancing how developers handle data absence, uninitialized variables, and the expression of "no value."
It solves the problem of absence
As we said, at its core, a null value represents the absence of a value or a "nothing" state within a variable. This concept is important when dealing with object references or pointer variables, where a clear distinction is necessary between having an uninitialized object and one that is intentionally empty or non-existent.
For instance, in object-oriented programming, when we talk about a null object, we're saying that a variable meant to hold a reference to an object currently doesn't point to any object at all in the computer's memory. This situation is notably different from having a variable that points to an existing object but doesn't have any data or attributes set yet—what you might call an "empty object."
It improves error handling and debugging
Null values also play a role in error handling and debugging. The ability to check for null values using null checks allows developers to prevent common errors such as the NullPointerException (NPE). This exception occurs when a program attempts to use an object reference that has been initialized to null, leading to crashes or unwanted behavior.
By intentionally using null values, programmers can design their code to gracefully handle scenarios where data might not be available or a reference is yet to be established. This approach makes software applications stronger and more dependable.
It makes optional and missing data representation easier
In many programming scenarios, especially in data-driven applications, it's crucial to represent the absence of data without resorting to arbitrary values. Null values enable this representation, allowing for a clear distinction between 'zero', 'empty string', and 'no value.'
This distinction is particularly important in databases, function arguments, and data processing, where the presence of a null value can significantly alter an application's logic and output.
It simplifies the interface and API design
The use of null values influences the design of interfaces and APIs, enabling the definition of optional parameters or return types. For instance, a function might return a null object or a null reference when the expected output cannot be generated due to the lack of input data or an error during processing.
This approach allows API consumers to implement conditional logic based on the presence or absence of a valid object, improving the interface's flexibility and usability.
What is the null reference exception?
Having explored the importance of null values in programming and their role, let's address a common challenge they present - the null reference exception.
This exception occurs when the code attempts to access or modify a property or method of an object that is currently null—in other words, when the program expects an object to exist at a certain memory location but finds nothing there.
Unlike a null reference, which is a deliberate use of null to indicate the absence of any object, a null reference exception is an error condition that arises from attempting to use a null as if it were a valid object.
This difference is crucial; while null references are used intentionally to represent the concept of "no object," null reference exceptions signal a mistake in the code, often a failure to properly check for null before performing operations on an object.
Learning how to handle these exceptions is important for avoiding unexpected crashes and keeping your code stable and error-free.
How do you get a null reference exception?
We will show examples of what can cause a NullReferenceException in a few languages that we use at Devōt:
Ruby on Rails and understanding "nil"
In Ruby, instead of null, the language utilizes nil to represent the absence of a value, aligning with the NilClass.
PHP and navigating the null value
PHP interprets a null value to signify the absence of any value or a variable that has been initialized with null.
JavaScript and dealing with "null"
JavaScript employs null to indicate that a pointer or reference variable intentionally points to a non-existent object, a critical concept for managing uninitialized variables and objects.
Is there a difference between a null pointer exception (NPE) and a null reference exception?
It's important to note that what is known as a null pointer exception in Java is essentially the same as a null reference exception in other programming languages, just under different names. In Java, it is referred to as a 'null pointer exception' (NPE) because, traditionally, languages like C use pointers to reference memory locations, and the term has carried over.
Java and NPE
The hidden trouble with null references
Null references can seem harmless at first, but they create significant challenges, particularly with the types of variables we use. We will try to answer by using a very basic example.
When null values surprise us
In dynamically typed programming languages, which include all the languages we use at Devōt, you might define a variable to hold a string value. This setup allows you to use string functions on that variable throughout the function where it's defined.
However, there may be instances where, instead of a string , the variable is assigned a null value—perhaps dynamically from the response of another function. From that point forward, any string-related operation on this variable will fail.
Another example is when you're calling a method on an object that was never initialized or trying to access an array element that doesn't exist. Essentially, you're trying to interact with something that simply isn't there. These situations lead to a NullReferenceException, though the specific name of the exception can vary across different programming languages.
How do you avoid null references?
Debugging the cause of a NullReferenceException is not always the easiest job. In fact, it might take a while to discover why a string variable has been assigned a NULL value, as in our example. The situation worsens when such an exception occurs in a production environment, as we're all aware of the high costs associated with spending excessive time on bug fixes. Because of that, it's important to write code that verifies whether a variable holds its expected value type and correctly handles scenarios when it's NULL. By doing so, we can minimize the time spent on debugging and focus more on developing new features.
Does null reference still give programmers a headache?
Despite being recognized as a billion-dollar mistake since its inception in 1965 and particularly after 2009, the question arises in 2024: Is the null reference still causing significant problems in programming, or have we, as developers, learned from those who came before us and become more aware in handling Null References during the development process?
Advances and challenges in handling null references
Initially, it's worth noting the advancements in programming languages like Kotlin, which identify Null References at the compilation stage, alerting developers with warnings before the application even runs.
Starting off, it's a positive note that certain programming languages, like Kotlin, can detect a Null Reference at the compilation stage, giving us a heads-up with a warning before the application even runs.
However, for many of the languages we frequently use at Devōt, such as Ruby, PHP, and JavaScript, compilers don't catch Null References until it's too late, and the application crashes, resulting in a NullReferenceException. Over time, developers have come up with clever strategies to avoid these exceptions. A popular approach is using if-else statements:
Another effective method is employing throw-catch mechanisms to manage NullReferenceExceptions. This makes error messages more user-friendly and simplifies the debugging process, making our lives a bit easier.
Innovations and solutions beyond simple fixes
Several programming languages have gone a step further, either by sidestepping the implementation of Null References altogether or by integrating mechanisms that ease the challenge they pose.
A standout example is the "void (null) safety mechanism" introduced by Bertrand Meyer in the Eiffel programming language. This innovative feature ensures that no reference can ever be null or void, providing a robust layer of safety. While Meyer spearheaded this approach, it's interesting to see other languages, like Kotlin and TypeScript (which we're quite familiar with at Devōt), adopting similar safety mechanisms independently.
On the other hand, not all languages sought to eliminate null references. Instead, some, like PHP8, embraced them through the introduction of Union Types. This approach doesn't avoid null references but integrates them more smoothly into the language's type system.
Union Types lets you clearly state that a variable can hold different types of values, including null. This makes it easier and safer to handle null values in your code. This flexibility is particularly useful in PHP programming, where handling various value types effectively can greatly improve code quality and readability.
With the variable's definition, as shown in the example above, we can assign both string and null to attribute2, and everything will remain valid.
Reevaluating the null reference dilemma
With these strategies and mechanisms in place, one might wonder if null references remain a concern. Despite the progress, the statistics of the Stack Overflow (the page we all know very well) paint a less optimistic picture. An analysis of questions tagged with nullreferenceexception reveals that, from 2008 to the present, there have been 2,786 such questions, with around 190 categorized as "Most Frequent." Some of these questions have garnered over 2 million views, indicating that a significant number of developers still grapple with Null Reference Exceptions.
We went further and did a little statistics on these results, and this is what we got:
Among the 2,779 questions, approximately 190 are tagged as "Most Frequent" on Stack Overflow, indicating these issues have been linked numerous times. This suggests that at least 190 times, a developer encountered a problem similar to one faced by another. Intriguingly, some questions from this subset have attracted over 2 million views, hinting that a significant number of developers encounter similar issues with NullReferenceException.
Recent questions, even those posted just a few days ago, have already attracted about 30 views each.
The most viewed question boasts 2,146,584 views, closely followed by another with 1,982,359 views. The total views for all questions in our analysis reached 12,074,710, underscoring the widespread nature of these queries. This level of engagement suggests a continuing challenge for many in dealing with NullReferenceExceptions.
It's worth mentioning that our findings are based solely on Stack Overflow questions tagged with 'null reference exception,' representing only 0.012% of all tagged questions (with the total being 23,897,356). Expanding our search to include various related tags or looking beyond Stack Overflow would likely reveal many more questions, but let's just say it's probably best not to go down that rabbit hole.
In conclusion, although this data represents just a small portion of all questions on Stack Overflow, it highlights two things: first, we're more aware of and have better tools for dealing with Null References now; second, we still struggle to debug and smoothly handle them without overusing if-else statements, which might not catch all Null Reference cases.
It's important to admit when you're wrong
Tony Hoare might have introduced something that will continue to challenge us for many years to come, but it's also possible to see him as an inspiration. Certainly, he's not the only one to have made a costly mistake; many major companies have made errors that were potentially much larger and more expensive. So, why regard him as an inspiration?
Because first of all, he recognized his mistake and openly admitted it to the world. It may have taken him 40 years, but admitting a mistake is commendable, no matter when it happens. Additionally, Tony Hoare has achieved something else noteworthy: the issue of null references has made us more cautious and attentive when assigning values to variables. Perhaps a less costly lesson would have been preferable, but things are as they are.
Aside from some interesting facts about null reference exceptions, I hope this blog reminds you that, in the end, the real fault lies not in making mistakes but in failing to own them.
