The Nebraska Problem in Open Source Software Development

: In the world of open source constructed on the basis of the Unix philosophy, there are cases of unsung heroic programs in obscure locations being maintained in a detailed way by a single unknown person or a small number of unknown people, mainly for personal reasons. However, if once these small programs close to the bottom rung of the ladder break, it may cause a loss of balance and collapse of our entire modern infrastructure. This is referred to as the Nebraska Problem in this paper. We can see from the actual and serious case of the Heartbleed bug that “the number of eyeballs” taken for granted in Linus’s Law up to this point needs to be proactively secured, and we need to consider complementary measures, such as SBOM, against risk in advance.


What is the Nebraska Problem?
The Nebraska problem is a term coined by this author. The idea for this was inspired by episode #2347 1 of xkcd, a web comic by Randall Munroe that is popular with hackers. 1 Although the date and time of this comic's release is not specified, judging from posts on the bulletin board site Reddit, (https://www.reddit.com/r/xkcd/comments/ibnqr3/xkcd_2347_depende ncy/) we can estimate that it was from 2020. The Munroe comic entitled "dependency" is shown in Figure 1, and the rectangular block represents the program development project for configuring the system. Upper-level blocks are placed on lower-level blocks and the programs constituting the upper level depend on the programs constituting the lower level. On the third level from the bottom, there is a thin block that supports the entire system of "all modern digital infrastructure." This thin block is "a project some random person in Nebraska has been thanklessly maintaining since 2003." 2 In other words, these programs, in obscure locations, are being maintained in detail by a single unknown person or a small number of unknown people, mainly for personal reasons, and if once these detailed programs, which are close to the bottom break, then the total balance of "all modern digital infrastructure" may be lost, causing it to collapse.
According to the caption, the illustrator drew the comic with an incident that occurred with ImageMagick in mind. ImageMagick is an image conversion utility released in 1990, and it is used for conversion between various graphics file formats, as well as executing various other types of conversions. There are numerous other libraries and APIs that are capable of performing these tasks, but as ImageMagick has been around for a long time, many programs, especially those used by web services, have relied on ImageMagick for its APIs or for performing the necessary conversions. As they are reliant on ImageMagick, a behind-the-scenes utility that has existed for a long time, their programs would break if ImageMagick were to disappear. Despite this being the case, ImageMagick is known to have numerous vulnerabilities (e.g., CVE Details, 2022), and in 2016, a 2 It is not clear why Nebraska was chosen, but it remains true that in U.S. popular culture, Nebraska is often treated as a synonym for "a country area with nothing to see or do" and has an unfavorable image. For example, the leading character in the popular TV drama "Better Call Saul" was last seen escaping to Omaha, Nebraska, pursued by both police and gangsters. major vulnerability (CVE-2016-3714) nicknamed ImageTragick was discovered. 3 Figure 1 depicts the existence of open source projects that have many bugs, even though they are widely used.

Unix Philosophy
The reason that "All modern digital infrastructure" is depicted with the structure shown in this comic is a result of the history of normal for one program to be used for one small task, with small programs being linked together by a shell script. This is the essence of the "Unix philosophy" (Raymond, 2004;Salus, 1994).
Additionally, in recent years, the JavaScript platform Node.js and Python have been widely used. These languages are designed in a way that developers from around the world can gather to use many small convenient libraries that each has written to make new, more powerful, libraries. The result of this, however, is that they have the risk of new unexpected bugs suddenly occurring somewhere in the chain of dependencies. In theory, such systems can be written and maintained with fewer lines of code. This may sound good for the developer, but a highly optimized system is also extremely susceptible to rapid change. For example, in the left-pad incident that occurred in 2016, a discontented developer, by introducing just 11 lines of code, was able to destroy everyone's build, and the builds of many web services dependent on left-pad failed (Avram, 2016;Collins, 2016) That is to say, because the code is open source and the source code is available publicly, a third party may modify the code and not notice the bug (due to high reusability and ease of modification). 4

Heartbleed Problem
The Heartbleed problem has been raised as a serious case of the highlighted a poor development support system that was considered insufficient given the importance of OpenSSL, especially its economic importance. The news at its time of discovery in 2014 led with the headline "The Internet Is Being Protected By Two Guys Named Steve." In fact, these guys named Steve were virtually unknown volunteers, whose efforts supported the security of major websites around the world, although they were overworked and underfunded.
Historically, the open source development model has been largely dependent on the free and continual contributions of hobbyists. Some projects, such as Linux, may be large enough to attract sufficient attention to warrant the creation of an organization; however, smaller projects may be reused by larger projects and effectively maintained by a single person or very few people. To maintain a library, it is necessary to have extensive knowledge not only of the library itself but also of its use cases and scope. This is normally only possible with several years of maintenance experience; therefore, the maintaining party cannot easily be replaced. OpenSSL (Marquess, 2014). As a result, there were no other developers engaged in the development of OpenSSL on a full-time basis other than the "two guys named Steve" (Marquess and Henson), and this may be one reason that the infiltration of the Heartbleed bug was overlooked. After the Heartbleed bug came to light, there was an increase in personal donations from young people, but this still amounted to no more than $9,000 a year.

Given Enough Eyeballs…
The the fact that OpenSSL source code was completely accessible, it still took two years or more for Heartbleed to be discovered.
Raymond's assertion was not only that this would take the place of software tests, but includes the idea that special effort is not required to detect bugs. It had been thought that open source project members would be able to notice and fix bugs during performing their duties as developers. However, what the Heartbleed problem brought to light was that even if the source code is published, the level of attention on it is low, which accounts for the lack of eyeballs.

Conclusion
The Unix philosophy has long been focused on as a methodology for software development. Indeed, the modularization of software development by linking small programs with minimal functionality and improving reusability through such modularization has been lauded as a good practice for software development. As a result, the Nebraska problem arose, in which a vulnerability in even a small piece of a large and complex system, which is developed on a voluntary basis, could compromise the entire system.
Although the author of the xkcd cartoon in Figure 1  "So today I filed a trivial GRUB bug (latest version runs out of memory, hardcoded heap size needs a bump). I just realized they haven't fixed *any* bug tracker bugs that weren't typos in the last 5 years. Seriously. I knew they were slow about releases, but wow." (Martin, 2022) That is to say, if we check the GRUB2 bug tracker, we can see that excepting trivial items, no major code additions or modifications have been made in the past five years. If we look at the Git commit log, the majority of the fix patches were sent from external companies or distro (abbreviation for distributers), and were just neglected after being sent, with maintenance only being carried out on a nominal basis.
To be more precise, development in open source is carried out separately at (1) the upstream level and (2) the distro level, and in some cases, due to the fact that they are not synchronized, (3) an effective fork is possible. In addition to this, it has been pointed out that whereas in the case of GRUB2, patches were actively applied at the Linux distro level, as the content of the patches differ depending on the distro, even if they are nominally the same version, there is a high likelihood that bugs fixed on a certain distro will remain on another distro, and that maintenance is only being carried out on a nominal basis. Put another way, even if there are a sufficient number of eyeballs, and even if some people have noticed and even fixed the problem, this does not mean it is possible to provide measures for all the code in the world.
As a complementary measure for this, the SBOM (Software Bill of Materials), which is a mandate for organizations that sell products to the federal government, and was contained in the Cybersecurity and dependencies included in a specific product. This means that any risk can be considered in advance.