![[Valid RSS]](images/valid-rss-rogers.png "Valid RSS")
This is a valid RSS feed.
This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.
... rel="self" type="application/rss+xml" />
^
line 39, column 0: (414 occurrences) [help]
line 408, column 0: (16 occurrences) [help]
<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" > <channel> <title>Radar</title> <atom:link href="https://www.oreilly.com/radar/feed/" rel="self" type="application/rss+xml" /> <link>https://www.oreilly.com/radar</link> <description>Now, next, and beyond: Tracking need-to-know trends at the intersection of business and technology</description> <lastBuildDate>Tue, 16 Apr 2024 10:22:09 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod> hourly </sy:updatePeriod> <sy:updateFrequency> 1 </sy:updateFrequency> <generator>https://wordpress.org/?v=5.3.17</generator> <item> <title>Attacking Supply Chains at the Source</title> <link>https://www.oreilly.com/radar/attacking-supply-chains-at-the-source/</link> <comments>https://www.oreilly.com/radar/attacking-supply-chains-at-the-source/#respond</comments> <pubDate>Tue, 16 Apr 2024 10:22:08 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[Security]]></category> <category><![CDATA[Software Engineering]]></category> <category><![CDATA[Signals]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15576</guid> <description><![CDATA[We’ve been very lucky. A couple of weeks ago, a supply-chain attack against the Linux xz Utils package, which includes the liblzma compression library, was discovered just weeks before the compromised version of the library would have been incorporated into the most widely used Linux distributions. The attack inserted a backdoor into sshd that would […]]]></description> <content:encoded><![CDATA[<p>We’ve been very lucky. A couple of weeks ago, a supply-chain attack against the Linux xz Utils package, which includes the liblzma compression library, was discovered just weeks before the compromised version of the library would have been incorporated into the most widely used Linux distributions. The attack inserted a backdoor into sshd that would have given threat actors remote shell access on any infected system.</p> <p>The details of the attack have been thoroughly discussed online. If you want a blow-by-blow exposition, here are <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://research.swtch.com/xz-timeline" target="_blank">two</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://boehs.org/node/everything-i-know-about-the-xz-backdoor" target="_blank">chronologies</a>. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/" target="_blank">ArsTechnica</a>, <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.schneier.com/blog/archives/2024/04/backdoor-in-xz-utils-that-almost-happened.html" target="_blank">Bruce Schneier</a>, and other sources have good discussions of the attack and its implications. For the purposes of this article, here’s a brief summary.</p> <p>The malware was introduced into xz Utils by one of its maintainers, an entity named Jia Tan. That’s almost certainly not a person’s name; the actual perpetrator is unknown. It’s likely that the attacker is a collective operating under a single name. Jia Tan began several years ago by submitting a number of changes and fixes to xz, which were included in the distribution, establishing a reputation for doing useful work. A coordinated attack against xz’s creator and maintainer, Lasse Collin, complained that Collin wasn’t approving patches quickly enough. This pressure eventually convinced him to add Jia Tan as a maintainer.</p> <p>Over two years, Jia Tan gradually added compromised source files to xz Utils. There’s nothing really obvious or actionable; the attackers were slow, methodical, and patient, gradually introducing components of the malware and disabling tests that might have detected the malware. There were no changes significant enough to attract attention, and the compromises were carefully concealed. For example, one test was disabled by the introduction of an innocuous single-character typo.</p> <p>Only weeks before the compromised xz Utils would have become part of the general release of RedHat, Debian, and several other distributions, Andrew Freund noticed some performance anomalies with the beta distribution he was using. He investigated further, discovered the attack, and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.openwall.com/lists/oss-security/2024/03/29/4" target="_blank">notified</a> the security community. Freund made it clear that he is not a security researcher, and that there may be other problems with the code that he did not detect.</p> <p>Is that the end of the story? The compromised xz Utils was never distributed widely, and never did any damage. However, many people remain on edge, with good reason. Although the attack was discovered in time, it raises a number of important issues that we can’t sweep under the rug:</p> <ul><li>We’re looking at a social engineering attack that achieves its aims by bullying—something that’s all too common in the Open Source world.</li><li>Unlike most supply chain attacks, which insert malware covertly by slipping it by a maintainer, this attack succeeded in inserting a corrupt maintainer, corrupting the release itself. You can’t go further upstream than that. And it’s <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openssf.org/blog/2024/04/15/open-source-security-openssf-and-openjs-foundations-issue-alert-for-social-engineering-takeovers-of-open-source-projects/" target="_blank">possible that other packages have been compromised</a> in the same way.</li><li>Many in the security community believe that the quality of the malware and the patience of the actors is a sign that they’re working for a government agency.</li><li>The attack was discovered by someone who wasn’t a security expert. The security community is understandably disturbed that they missed this.</li></ul> <p>What can we learn from this?</p> <p>Everyone is responsible for security. I’m not concerned that the attack wasn’t discovered by the a security expert, though that may be somewhat embarrassing. It really means that everyone is in the security community. It’s often said “Given enough eyes, all bugs are shallow.” You really only need one set of eyeballs, and in this case, those eyeballs belonged to Andres Freund. But that only begs the question: how many eyeballs were watching? For most projects, not enough—possibly none. If you notice something that seems funny, look at it more deeply (getting a security expert’s help if necessary); don’t just assume that everything is OK. “If you see something, say something.” That applies to corporations as well as individuals: don’t take the benefits of open source software without committing to its maintenance. Invest in ensuring that the software we share is secure. The Open Source Security Foundation (OpenSSF) <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openssf.org/blog/2024/04/15/open-source-security-openssf-and-openjs-foundations-issue-alert-for-social-engineering-takeovers-of-open-source-projects/" target="_blank">lists</a> some suspicious patterns, along with best practices to secure a project.</p> <p>It’s more concerning that a particularly abusive flavor of social engineering allowed threat actors to compromise the project. As far as I can tell, this is a new element: social engineering usually takes a form like “Can you help me?” or “I’m trying to help you.” However, many open source projects tolerate abusive behavior. In this case, that tolerance opened a new attack vector: badgering a maintainer into accepting a corrupted second maintainer. Has this happened before? No one knows (yet). Will it happen again? Given that it came so close to working once, almost certainly. Solutions like screening potential maintainers don’t address the real issue. The kind of pressure that the attackers applied was only possible because that kind of abuse is accepted. That has to change.</p> <p>We’ve learned that we know much less about the integrity of our software systems than we thought. We’ve learned that supply chain attacks on open source software can start very far upstream—indeed, at the stream’s source. What we need now is to make that fear useful by looking carefully at our software supply chains and ensuring their safety—and that includes social safety. If we don’t, next time we may not be so lucky.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/attacking-supply-chains-at-the-source/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Quality Assurance, Errors, and AI</title> <link>https://www.oreilly.com/radar/quality-assurance-errors-and-ai/</link> <comments>https://www.oreilly.com/radar/quality-assurance-errors-and-ai/#respond</comments> <pubDate>Tue, 09 Apr 2024 09:58:34 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Signals]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15570</guid> <description><![CDATA[A recent article in Fast Company makes the claim “Thanks to AI, the Coder is no longer King. All Hail the QA Engineer.” It’s worth reading, and its argument is probably correct. Generative AI will be used to create more and more software; AI makes mistakes and it’s difficult to foresee a future in which […]]]></description> <content:encoded><![CDATA[<p>A recent article in <em>Fast Company</em> makes the claim “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.fastcompany.com/91045570/thanks-to-ai-the-coder-is-no-longer-king-all-hail-the-qa-engineer" target="_blank">Thanks to AI, the Coder is no longer King. All Hail the QA Engineer</a>.” It’s worth reading, and its argument is probably correct. Generative AI will be used to create more and more software; AI makes mistakes and it’s difficult to foresee a future in which it doesn’t; therefore, if we want software that works, Quality Assurance teams will rise in importance. “Hail the QA Engineer” may be clickbait, but it isn’t controversial to say that testing and debugging will rise in importance. Even if generative AI <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://info.deeplearning.ai/robots-talk-back-ai-security-risks-political-deepfakes-pretrained-models-on-the-cheap" target="_blank">becomes much more reliable</a>, the problem of finding the “last bug” will never go away.</p> <p>However, the rise of QA raises a number of questions. First, one of the cornerstones of QA is testing. Generative AI can generate tests, of course—at least it can generate unit tests, which are fairly simple. Integration tests (tests of multiple modules) and acceptance tests (tests of entire systems) are more difficult. Even with unit tests, though, we run into the basic problem of AI: it can generate a test suite, but that test suite <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/library/view/use-chatgpt-to/9781098167646/ch01.html" target="_blank">can have its own errors</a>. What does “testing” mean when the test suite itself may have bugs? Testing is difficult because good testing goes beyond simply verifying specific behaviors.</p> <p>The problem grows with the complexity of the test. Finding bugs that arise when integrating multiple modules is more difficult and becomes even more difficult when you’re testing the entire application. The AI might need to use <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.selenium.dev/" target="_blank">Selenium</a> or some other test framework to simulate clicking on the user interface. It would need to anticipate how users might become confused, as well as how users might abuse (unintentionally or intentionally) the application.</p> <p>Another difficulty with testing is that bugs aren’t just minor slips and oversights. The most important bugs result from misunderstandings: misunderstanding a specification or correctly implementing a specification that doesn’t reflect what the customer needs. Can an AI generate tests for these situations? An AI might be able to read and interpret a specification (particularly if the specification was written in a machine-readable format—though that would be another form of programming). But it isn’t clear how an AI could ever evaluate the relationship between a specification and the original intention: what does the customer really want? What is the software really supposed to do?</p> <p>Security is yet another issue: is an AI system able to red-team an application? I’ll grant that AI should be able to do an excellent job of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Fuzzing" target="_blank">fuzzing</a>, and we’ve seen game playing AI <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theverge.com/tldr/2018/2/28/17062338/ai-agent-atari-q-bert-cracked-bug-cheat" target="_blank">discover “cheats.”</a> Still, the more complex the test, the more difficult it is to know whether you’re debugging the test or the software under test. We quickly run into an extension of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.laws-of-software.com/laws/kernighan/" target="_blank">Kernighan’s Law</a>: debugging is twice as hard as writing code. So if you write code that’s at the limits of your understanding, you’re not smart enough to debug it. What does this mean for code that you haven’t written? Humans have to test and debug code that they didn’t write all the time; that’s called “maintaining legacy code.” But that doesn’t make it easy or (for that matter) enjoyable.</p> <p>Programming culture is another problem. At the first two companies I worked at, QA and testing were definitely not high-prestige jobs. Being assigned to QA was, if anything, a demotion, usually reserved for a good programmer who couldn’t work well with the rest of the team. Has the culture changed since then? Cultures change very slowly; I doubt it. Unit testing has become a widespread practice. However, it’s easy to write a test suite that give good coverage on paper, but that actually tests very little. As software developers realize the value of unit testing, they begin to write better, more comprehensive test suites. But what about AI? Will AI yield to the “temptation” to write low-value tests?</p> <p>Perhaps the biggest problem, though, is that prioritizing QA doesn’t solve the problem that has plagued computing from the beginning: programmers who never understand the problem they’re being asked to solve well enough. Answering a Quora question that has nothing to do with AI, Alan Mellor <a href="https://www.quora.com/Why-is-there-a-sudden-craze-of-programmers-with-little-math-background-jumping-onto-machine-learning-which-requires-a-much-different-skill-set-than-traditional-programming" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">wrote</a>:</p> <blockquote class="wp-block-quote"><p>We all start programming thinking about mastering a language, maybe using a design pattern only clever people know.</p><p>Then our first real work shows us a whole new vista.</p><p>The language is the easy bit. The problem domain is hard.</p><p>I’ve programmed industrial controllers. I can now talk about factories, and PID control, and PLCs and acceleration of fragile goods.</p><p>I worked in PC games. I can talk about rigid body dynamics, matrix normalization, quaternions. A bit.</p><p>I worked in marketing automation. I can talk about sales funnels, double opt in, transactional emails, drip feeds.</p><p>I worked in mobile games. I can talk about level design. Of one way systems to force player flow. Of stepped reward systems.</p><p>Do you see that we have to learn about the business we code for?</p><p>Code is literally nothing. Language nothing. Tech stack nothing. Nobody gives a monkeys [sic], we can all do that.</p><p>To write a real app, you have to understand why it will succeed. What problem it solves. How it relates to the real world. Understand the domain, in other words.</p></blockquote> <p>Exactly. This is an excellent description of what programming is really about. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/fearing-the-wrong-thing/" target="_blank">Elsewhere</a>, I’ve written that AI might make a programmer 50% more productive, though this figure is probably optimistic. But programmers only spend about 20% of their time coding. Getting 50% of 20% of your time back is important, but it’s not revolutionary. To make it revolutionary, we will have to do something better than spending more time writing test suites. That’s where Mellor’s insight into the nature of software so crucial. Cranking out lines of code isn’t what makes software good; that’s the easy part. Nor is cranking out test suites, and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://read.engineerscodex.com/p/metas-new-llm-based-test-generator" target="_blank">if generative AI can help write tests</a> without compromising the quality of the testing, that would be a huge step forward. (I’m skeptical, at least for the present.) The important part of software development is understanding the problem you’re trying to solve. Grinding out test suites in a QA group doesn’t help much if the software you’re testing doesn’t solve the right problem.</p> <p>Software developers will need to devote more time to testing and QA. That’s a given. But if all we get out of AI is the ability to do what we can already do, we’re playing a losing game. The only way to win is to do a better job of understanding the problems we need to solve.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/quality-assurance-errors-and-ai/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>AI Has an Uber Problem</title> <link>https://www.oreilly.com/radar/ai-has-an-uber-problem/</link> <comments>https://www.oreilly.com/radar/ai-has-an-uber-problem/#respond</comments> <pubDate>Thu, 04 Apr 2024 16:51:46 +0000</pubDate> <dc:creator><![CDATA[Tim O’Reilly]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Research]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15537</guid> <description><![CDATA[“The economic problem of society…is a problem of the utilization of knowledge which is not given to anyone in its totality.” —Friedrich A. Hayek, “The Use of Knowledge in Society” Silicon Valley venture capitalists and many entrepreneurs espouse libertarian values. In practice, they subscribe to central planning: Rather than competing to win in the marketplace, […]]]></description> <content:encoded><![CDATA[<div class="wp-block-group has-very-light-gray-background-color has-background"><div class="wp-block-group__inner-container"><p class="has-text-align-center">This article originally appeared in <em><a href="https://www.theinformation.com/articles/ai-has-an-uber-problem" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">The Information</a></em> on March 5th, 2024.</p></div></div> <p></p> <blockquote class="wp-block-quote"><p><em>“The economic problem of society…is a problem of the utilization of knowledge which is not given to anyone in its totality.” </em></p><cite><em>—Friedrich A. Hayek, “</em><a href="https://www.cato.org/sites/cato.org/files/articles/hayek-use-knowledge-society.pdf" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)"><em>The Use of Knowledge in Society</em></a><em>”</em><br></cite></blockquote> <p>Silicon Valley venture capitalists and many entrepreneurs espouse libertarian values. In practice, they subscribe to central planning: Rather than competing to win in the marketplace, entrepreneurs compete for funding from the Silicon Valley equivalent of the Central Committee. The race to the top is no longer driven by who has the best product or the best business model, but by who has the blessing of the venture capitalists with the deepest pockets—a blessing that will allow them to acquire the most customers the most quickly, often by providing services below cost. Reid Hoffman called this pattern “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.blitzscaling.com/" target="_blank">blitzscaling</a>,” claiming in the subtitle of his book with that name that it is “The Lightning-Fast Path to Building Massively Valuable Companies.”</p> <p>I disagree. It is a dark pattern, a map to suboptimal outcomes rather than the true path to competition, innovation and the creation of robust companies and markets. As Bill Janeway noted in his critique of the capital-fueled bubbles that resulted from the ultra-low interest rates of the decade following the 2007–2009 financial crisis, “<a href="https://www.billjaneway.com/capital-is-not-a-strategy" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">capital is not a strategy</a>.”</p> <p>Venture capitalists don’t have a crystal ball. To the extent that entrepreneurial funding is more concentrated in the hands of a few, private finance can drive markets independent of consumer preferences and supply dynamics. Market discipline is significantly delayed—until the initial public offering or later. And of course, today IPOs <em>are</em> delayed, often precisely because companies can get all the capital they need from a small number of deep-pocketed investors. Founders and employees are even able to cash out some of their shares without having to face the scrutiny of public markets, much as if bettors on a horse race could take their money off the table as the horses round the first turn. Thus, far from finance being an extension of the market (with lots of independent signals aggregated to ensure competition and consumer choice), capital can ignore the will of the market. </p> <p>The ride-hailing business offers a classic example of the distortive over-reliance on capital rather than consumer choice. It began with bold prophecies of ride-hailing replacing not just taxis but all private vehicles, and ended with a national duopoly of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.consumerreports.org/personal-finance/uber-vs-taxi-which-is-cheaper/" target="_blank">on-demand taxis at prices no better and often worse </a>than those of the previous over-regulated local taxi market. In a well-functioning market, many startups would have explored a technology innovation like on-demand transportation over a much longer period. In that alternate history, entrepreneurs would have competed with different pricing strategies, different rate structures for drivers and perhaps even completely different business models. Eventually, those that survived would have done so because they were delivering the service chosen by the most customers and the most drivers. That is true product-market fit. </p> <p>But in the Central Committee version of Silicon Valley, Uber and Lyft, backed by billions of dollars of venture capital, drove out the competition rather than defeating it, subsidizing customer acquisition and an unsustainable business model—and in the case of Uber, continuing to attract new capital with promises of speculative future cost savings via self-driving cars. Instead, once the market had consolidated, Uber and Lyft only reached profitability through massive price increases. What might have happened if there had been true competition in this market? We will never know. </p> <p>By contrast, during the dot-com bubble, most companies consumed tiny amounts of capital by today’s standards. The funding was spread across thousands of companies, and it took a decade or more of relentless innovation and competition for the industry to become dangerously concentrated. This is a classic example of what Janeway calls a “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.billjaneway.com/productive-bubbles" target="_blank">productive bubble</a>.” Remarkably, most of the winning companies were profitable in only a few years, and eventually they became hugely profitable. Google raised only $36 million in venture capital on its way to dominance. Facebook raised billions, but it did so only <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://qz.com/1540608/the-problem-with-silicon-valleys-obsession-with-blitzscaling-growth" target="_blank">to fund faster growth for a business model</a> that insiders have told me was very close to profitable the entire time. They weren’t buying users with subsidized prices; they were building data centers. Even Amazon, long unprofitable, took in very little investment capital, instead funding itself with debt supported by a business model that produced previously unprecedented levels of free cash flow.</p> <p>To be sure, sometimes companies do require a lot of capital to lay the groundwork for a possible future. Tesla and SpaceX are good examples. They used their funding to do serious research and development, to build factories, cars, batteries, rockets and satellites. This is using capital properly: to fund the hard costs associated with creating something new until the projected unit economics lead to a self-sustaining business. It’s also worth noting that in those cases private funding was powerfully augmented by state support: carbon credits and electric vehicle incentives for Tesla, and NASA progress payments for SpaceX.</p> <p>That kind of investment was unnecessary in the case of ride-hailing. The startups simply used the money to amass market power by subsidizing blitzscaled growth. Others had already deployed the capital to build much of the infrastructure for ride-hailing—GPS satellites and GPS-enabled smartphones. Even the innovation of using GPS to match passengers and drivers was not developed by the VC-backed market leaders, but by <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://sunilpaul.medium.com/so-long-sidecar-and-thanks-74c8a0955064" target="_blank">the true market pioneer, Sidecar</a>, which was quickly sidelined when it failed to raise enough capital to gain a leading share in the market it had first envisioned. </p> <p>In the case of artificial intelligence, training large models is indeed expensive, requiring large capital investments. But those investments demand commensurately large returns. The investors who pile billions of dollars into a huge bet are expecting not just to be paid back, but paid back a hundredfold. The capital-fueled race to build the largest models has already led to bad behavior. OpenAI, for example, has trained not just on publicly available data but reportedly on <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theatlantic.com/technology/archive/2023/09/books3-database-generative-ai-training-copyright-infringement/675363/" target="_blank">copyrighted content retrieved from pirate sites</a>. This has led to lawsuits and settlements. But even those settlements are likely to be bad for the development of a healthy entrepreneurial ecosystem. As <a href="https://www.oreilly.com/radar/the-openai-endgame/" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">Mike Loukides points out</a>, “Smaller startups…will be priced out, along with every open-source effort. By settling, OpenAI will eliminate much of their competition.”</p> <p>Meanwhile, the largest models’ absorption of all content into “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Borg" target="_blank">the Borg</a>” of AI data will eliminate opportunities for the owners of specialized content repositories to profit from their own work. Innovators are already finding that much can be done at lower cost with smaller, more targeted open-source models. They can fine-tune these smaller models for specific problem domains, allowing trusted content providers (like my own company’s <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/online-learning/feature-answers.html" target="_blank">O’Reilly Answers</a> and related AI-generated services) to profit from our own expertise.</p> <p>OpenAI is making an effort to create a platform on which entrepreneurs can build vertical applications, but only if they pay tribute to the centralized business model in the form of API fees. OpenAI is also skimming the cream, quickly dominating some of the most profitable categories—image generation, video generation, speech synthesis, computer programming—that in a well-functioning market would be explored by dozens or hundreds of competing efforts, until one or two find the winning combination of product and business model. If entrepreneurs discover other profitable categories, giants such as OpenAI will move quickly to dominate these as well. </p> <p>The capital-fueled AI land grab is of course only one axis of premature market concentration. As Max von Thun points out in “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.techpolicy.press/monopoly-power-is-the-elephant-in-the-room-in-the-ai-debate/" target="_blank">Monopoly Power Is the Elephant in the Room in the AI Debate</a>,” much of the investment to train models is coming in the form of strategic partnerships (including both cloud computing credits and potential revenue deals) with existing industry giants Microsoft, Amazon and Google (and in the case of open-source models, Meta Platforms). As von Thun notes, “These partnerships appear to be serving the same purpose as ‘killer acquisitions’ in the past—think of Facebook’s acquisition of WhatsApp or Google’s purchase of YouTube—raising serious concerns about fair competition in the fledgling AI market.” The risk of these deals is, again, that a few centrally chosen winners will quickly emerge, meaning there’s a shorter and less robust period of experimentation.</p> <p>And, at least based on <a rel="noreferrer noopener" href="https://www.theinformation.com/articles/anthropics-gross-margin-flags-long-term-ai-profit-questions" target="_blank">recent reporting by The Information about Anthropic’s operating margins</a>, it may be that, like Uber and Lyft, the overfunded AI market leaders may only be able to deliver on investors’ heated expectations by crushing all competition. That’s not betting on the wisdom of the market and what Hayek called “the utilization of knowledge which is not given to anyone in its totality.” That’s betting on premature consolidation and the wisdom of a few large investors to choose a future everyone else will be forced to live in.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/ai-has-an-uber-problem/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Radar Trends to Watch: April 2024</title> <link>https://www.oreilly.com/radar/radar-trends-to-watch-april-2024/</link> <comments>https://www.oreilly.com/radar/radar-trends-to-watch-april-2024/#respond</comments> <pubDate>Tue, 02 Apr 2024 10:16:22 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[Radar Trends]]></category> <category><![CDATA[Signals]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15555</guid> <description><![CDATA[There are lots of new models, including one from Apple, but that’s hardly news. AI news is infiltrating other sections of Trends (particularly Programming and Security)—but that’s also hardly news. NVIDIA CEO Jensen Huang has said that AI will replace coding—but again, he’s not the first. But what’s new is Devin: an AI software engineer […]]]></description> <content:encoded><![CDATA[<p>There are lots of new models, including one from Apple, but that’s hardly news. AI news is infiltrating other sections of Trends (particularly Programming and Security)—but that’s also hardly news. NVIDIA CEO Jensen Huang has said that AI will replace coding—but again, he’s not the first. But what’s new is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.cognition-labs.com/introducing-devin" target="_blank">Devin</a>: an AI software engineer from Cognition Labs. Its makers claim that it “can execute complex engineering tasks requiring thousands of decisions. Devin can recall relevant context at every step, learn over time, and fix mistakes.” Devin is in early access; what we’ve heard from those who’ve used it is that it’s far from finished—but even in that state, it’s very impressive. There’s also an open source <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/OpenDevin/OpenDevin" target="_blank">OpenDevin</a> project on GitHub.</p> <p>A supply chain attack added a back door to Linux systems through the widely used xz package. Fortunately, this attack was discovered before the package was incorporated into the leading Linux distributions. However, the attack raises a lot of troubling questions about security—including the question of how we know software is trustworthy. The attack wasn’t discovered by security experts analyzing the code; social engineering may even have been used to prevent it from being tested adequately. The attack was discovered by an engineer who noticed some performance anomalies. Nobody knows who the maintainer who inserted the back door really is; it may not be a single person. This time, we were lucky.</p> <h2>Artificial Intelligence</h2> <ul><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2024/03/25/1090111/tech-industry-open-source-ai-definition-problem/" target="_blank">What does open source mean for AI?</a> What does it include aside from code? Can there be restrictions on how the AI is used? Nobody knows. And it’s entirely too likely that the vacuum will be filled by a self-interested definition coming from one of the internet giants.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://jan.ai/" target="_blank">Jan</a> is a new way of packaging open source language models so that they can be run completely locally. It’s available for Windows, macOS, and Linux.</li><li>Can <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/the-advent-of-automated-observability/" target="_blank">AI be an aid to observability</a>? Yes—both by recognizing normal and abnormal activity and by interpreting and summarizing log data and making suggestions for resolving problems.</li><li>NVIDIA has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/information-technology/2024/03/nvidia-announces-moonshot-to-create-embodied-human-level-ai-in-robot-form/" target="_blank">announced</a> that they intend to create an “embodied” AI: an AI incorporated into a humanoid robotic form. Is this “moonshot” just an attempt at realized science fiction or an important step on the road to general intelligence?</li><li>At NVIDIA’s developer conference, their CEO outlined a vision for the <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techwireasia.com/03/2024/nvidias-ceo-jensen-huang-ai-will-take-over-coding-making-learning-optional/" target="_blank">future of programming</a> in which AI systems replace the entire development pipeline. Humans remain in the loop and in control, but they will only use natural human languages.</li><li>The <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/albertan017/LLM4Decompile" target="_blank">LLM4Decompile</a> project is building large language models for <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2403.05286" target="_blank">decompiling</a> software (translating from assembly back into a higher level language like C). This would be a tremendous tool for reverse engineering. The models are available on <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/arise-sustech" target="_blank">Hugging Face</a>.</li><li>Now Apple has a large language model. It isn’t open to the public, but they’ve published a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2403.09611" target="_blank">paper</a> about it. In a Twitter post (that I can’t find) they claim performance similar to Gemini-1 at each model size.</li><li>Answer.ai is releasing an <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html" target="_blank">open source system for fine tuning large language models</a> with up to 70B parameters. It can run on a desktop computer with two commodity gaming GPUs.</li><li>A month or two ago, we noted that attackers have proven that they can jailbreak large language models by using steganographic techniques to hide hostile prompts within an image. It turns out you don’t have to be subtle: <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.tomshardware.com/tech-industry/artificial-intelligence/researchers-jailbreak-ai-chatbots-with-ascii-art-artprompt-bypasses-safety-measures-to-unlock-malicious-queries" target="_blank">ASCII art</a> that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2024/03/researchers-use-ascii-art-to-elicit-harmful-responses-from-5-major-ai-chatbots/" target="_blank">spells out the hostile words</a> will suffice.</li><li>SudoLang is a programming language for <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://medium.com/javascript-scene/sudolang-a-powerful-pseudocode-programming-language-for-llms-d64d42aa719b" target="_blank">interacting with large language models</a>. It’s not the only attempt along these lines; we’ve also noted GPTScript. SudoLang is particularly interesting, though, because the language was designed in part by GPT-4.</li><li>Simon Willison clarifies the distinction between <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://simonwillison.net/2024/Mar/5/prompt-injection-jailbreaking/" target="_blank">prompt injection and jailbreaking</a>. Prompt injection involves concatenating trusted and untrusted input in prompts. It is far more dangerous than jailbreaking and harder (perhaps impossible) to defend against.</li><li>A generative AI platform called <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.loremachine.ai/" target="_blank">Lore Machine</a> can take a short story and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2024/03/05/1089458/generative-ai-turn-my-story-into-comic-images-lore-machine/" target="_blank">turn it into an illustrated comic</a>.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2310.17389" target="_blank">ToxicChat</a> is a new benchmark for detecting toxic prompts sent to language models. It is based on actual prompts collected by language models rather than social media content.</li><li>Anthropic’s <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.anthropic.com/news/claude-3-family" target="_blank">latest series of models</a>, Claude 3, is now available. The most advanced model, Opus, is only available through subscription. All of them feature a 200,000-token context window.</li><li>Over the past few years, large models have reduced their data requirements by going from 32-bit floating point to 8 bits to 4 bits, in a process called “quantization.” The next step forward is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2402.17764" target="_blank">single-bit models</a> (actually, 1.58 bits).</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/gptscript-ai/gptscript" target="_blank">GPTScript</a> is a simple programming language for automating interactions with GPT. It’s starting to look a lot like a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/formal-informal-languages/" target="_blank">formal informal</a> language.</li></ul> <h2>Programming</h2> <ul><li>GitHub now offers <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/githubs-new-ai-powered-tool-auto-fixes-vulnerabilities-in-your-code/" target="_blank">Code Scanning Autofix</a>, a service that uses AI to detect vulnerabilities and suggest fixes to code written in Java, JavaScript, Python, and TypeScript. They claim that it can detect and correct over 90% of known vulnerabilities. They note that it’s still important for the programmer to verify that the suggestion actually fixes the vulnerability.</li><li>JetBrains now <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/jetbrains-launches-ci-cd-tool-for-small-mid-sized-dev-teams/" target="_blank">offers</a> <a rel="noreferrer noopener" aria-label="TeamCity Pipelines (opens in a new tab)" href="https://www.jetbrains.com/teamcity/pipelines/" target="_blank">TeamCity Pipelines</a>, a CI/CD tool for small to midsized teams. It is currently in public beta. Simpler tools that solve the problems of smaller projects are a welcome addition to the tooling scene.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/dibyendumajumdar/ravi" target="_blank">Ravi</a> is a new dialect of Lua that supports optional static typing. It has a just-in-time compiler and can also compile directly to machine code.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://boinc.berkeley.edu/index.php" target="_blank">BOINC</a> is a project that lets you allow scientific computing tasks to run on your computer in the background. It’s similar to projects like <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://setiathome.berkeley.edu/" target="_blank">SETI@Home</a>, but more general; it isn’t associated with a specific research project. BOINC is based at UC Berkeley and supported by the NSF. </li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.cognition-labs.com/" target="_blank">Devin</a> is “the world’s first fully autonomous AI software engineer.” The claims made for Devin are impressive: it can learn new technologies from a blog post, build deploy apps, fix bugs, train language models, and more. If it lives up to these claims, it will be very impressive.</li><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.zama.ai/" target="_blank">startup</a> has released open source libraries for <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/zama-ai" target="_blank">fully homomorphic encryption</a>. Homomorphic encryption is a set of codes and protocols for computing with encrypted data without first decrypting the data.</li><li>We know that language models can assist in writing code. Can they also <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/are-copilots-ready-to-provision-your-infrastructure/" target="_blank">assist in building infrastructure as code</a>?</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2024/02/github-besieged-by-millions-of-malicious-repositories-in-ongoing-attack/" target="_blank">GitHub is being attacked</a> by cybercriminals who are creating millions of repositories containing malware. The malicious repos have names similar to legitimate repos in hopes that programmers will use the wrong repo (often with the encouragement of social engineering). </li><li>Github is offering <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2024-02-microsoft-github-companies-souped-ai.html" target="_blank">Copilot Enterprise</a>, a higher-priced version of Copilot that knows about a company’s codebase. Code completions are based on code in the company’s repositories, so they match the company’s practices. It can even learn proprietary in-house languages.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://waxjs.net/features/" target="_blank">Wax</a> is an open source framework for building word processing software on the web. It facilitates change tracking, commenting, equations, basic text styling, managing citations, and other features you’d expect in a professional word processing system.</li></ul> <h2>Operations</h2> <ul><li>Brendan Gregg has posted a list of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.brendangregg.com/blog/2024-03-24/linux-crisis-tools.html" target="_blank">Linux crisis tools</a>: utilities that you are likely to need to diagnose and fix an outage and that your favorite distribution might not have.</li><li>DBOS is a new <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/meet-dbos-a-database-alternative-to-kubernetes/" target="_blank">cloud native operating system</a> that is based on a high performance distributed database. It is intended to replace the Linux/Kubernetes combination that has become the basis for orchestrating complex distributed applications.</li><li>Buoyant is now <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/some-linkerd-users-must-pay-fear-and-anger-explained/" target="_blank">charging</a> organizations with 50 or more users for access to the latest stable release of the linkerd service mesh. They have not changed linkerd’s licensing, which is still open source (Apache 2.0).</li><li>Netflix has released <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://netflixtechblog.com/announcing-bpftop-streamlining-ebpf-performance-optimization-6a727c1ae2e5" target="_blank">bpftop</a>, a command line tool for monitoring programs that use eBPF (extended Berkeley packet filters). bpftop gives users insight into their eBPF tools, preventing eBPF from inadvertently compromising performance while attempting to improve performance.</li></ul> <h2>Web</h2> <ul><li>Facebook’s Threads now <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.threads.net/@zuck/post/C4yMgnQRH_T?hl=en" target="_blank">allows</a> users to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.engadget.com/a-threads-beta-lets-you-share-to-mastodon-and-other-fediverse-services-194732036.html" target="_blank">share</a> their posts on Mastodon. The feature is currently opt-in. Threads users can’t yet view posts made by Mastodon users. How Facebook will handle Mastodon users’ private data and dislike of advertising remains to be seen.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/paveldedik/ludic" target="_blank">Ludic</a> is a new lightweight web framework that is built to be used with <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://htmx.org/" target="_blank">htmx</a>. It uses a component approach similar to React but does not require any JavaScript to build dynamic applications. It is based on Python 3.12.</li><li>YouTube is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blog.google/intl/en-in/products/platforms/how-were-helping-creators-disclose-altered-or-synthetic-content/" target="_blank">requiring</a> creators to disclose when they have used generative AI to create or modify otherwise realistic video. This rule does not apply to content that is “clearly unrealistic” (e.g., animations), color adjustments or beauty filters, and background effects (e.g., blur).</li><li>LaVague is a large language model designed for <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/lavague-ai/LaVague" target="_blank">controlling browser interactions</a>. It would be ideal for controlling a testing framework like Selenium. And it may be useful for automating other “mundane tasks.”</li><li>The Bluesky social network, created by Twitter founder Jack Dorsey, now allows <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techcrunch.com/2024/02/22/bluesky-opens-up-federation-letting-anyone-run-their-own-server/?guccounter=1&guce_referrer=aHR0cHM6Ly9mZWVkbHkuY29tLw&guce_referrer_sig=AQAAAISTxB1YMxSqj9qzkBxwH4WHw3QNiPSv3ypZx4hv9IdZvk4-i-VFRlpK4X8EtA40eSBFO9XTq0Vhk1zeCUDvKKfcTBfutWh8swJd6D-q9CPtxEbLUhIK2CexlfkqwMCYB7iGeM-Kq3R-_ELfzHrLdpe1rI3TkJPvNJW0OHuY7QiA" target="_blank">federation</a>: individuals and groups can now run their own servers, similar to Mastodon.</li></ul> <h2>Security</h2> <ul><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/" target="_blank">supply-chain compromise</a> added a backdoor to Linux’s xz package. The <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/red-hat-warns-of-backdoor-in-xz-tools-used-by-most-linux-distros/" target="_blank">attack</a> raises many questions. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://social.treehouse.systems/@megmac/112186436680099413" target="_blank">Social engineering</a> may have prevented Google from testing it adequately; the maintainer was pressured into adding a second maintainer who was probably the attacker and may have been state-sponsored; and it was discovered because of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://mastodon.murkworks.net/@moira/112186986472166118" target="_blank">performance anomalies</a>.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/new-loop-dos-attack-may-impact-up-to-300-000-online-systems/" target="_blank">Loop DoS</a> is a new denial of service attack in which targeted computers send UDP packets back and forth in an infinite loop. Equipment from several major vendors, including Cisco, Microsoft, and Broadcom, is reported to be vulnerable.</li><li>A new attack against LLMs allows attackers to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2024/03/hackers-can-read-private-ai-assistant-chats-even-though-theyre-encrypted/" target="_blank">recover the texts of chat sessions</a> even if they are encrypted. The attack is based on observing the lengths of the tokens and matching the lengths to words. This vulnerability applies to all LLMs except for Google’s Gemini.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2024/01/new-uefi-vulnerabilities-send-firmware-devs-across-an-entire-ecosystem-scrambling/" target="_blank">Pixieboot</a> (aka PXE boot) is a collection of attacks against UEFI firmware, a very low-level system-within-a-system that controls the boot process on most modern PCs. While this particular set of vulnerabilities is mostly of concern to cloud and datacenter operators, Cory Doctorow <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://doctorow.medium.com/demon-haunted-computers-are-back-baby-bf06530a25cb" target="_blank">writes</a> about the danger of nonupdateable subsystems that treat the user as a threat.</li><li>Cloudflare is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theregister.com/2024/03/05/cloudflare_firewall_ai/" target="_blank">introducing</a> an AI firewall product that, among other things, will eventually include a prompt validation feature that will detect and block prompt injection attacks. The feature may help with jailbreaking (a single hostile prompt), but it’s harder to see how it would be effective against true prompt injection (a hostile prompt concatenated with a trustworthy prompt).</li><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/pdf/2311.16119.pdf" target="_blank">paper</a> analyzes over 600,000 prompt injection attacks to produce a taxonomy of vulnerabilities. The authors collected the attacks by running a global prompt hacking competition.</li><li>Docker, Confluence, Redis, and Apache Yarn are being targeted by malware <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/hackers-target-docker-hadoop-redis-confluence-with-new-golang-malware/" target="_blank">in a new set of attacks</a>. The malware is written in Go, though it is clumsily disguised to look like shell scripts.</li><li>Even more prompt injection attacks: Microsoft Copilot (distinct from Github Copilot) is vulnerable to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://embracethered.com/blog/posts/2024/whoami-conditional-prompt-injection-instructions/" target="_blank">conditional prompt injection</a> attacks, where the hostile prompt is activated only for a specific user.</li><li>Yes, there’s now a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.schneier.com/blog/archives/2024/03/llm-prompt-injection-worm.html" target="_blank">prompt injection</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://drive.google.com/file/d/1pYUm6XnKbe-TJsQt2H0jw9VbT_dO6Skk/view" target="_blank">worm</a>. A hostile prompt is embedded in an email, which then gets sent to the AI-based email assistant through RAG. Along with stealing data, the prompt can instruct the email assistant to generate new emails that spread the worm.</li></ul> <h2>Things</h2> <ul><li>Another <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.velo.ai/product/copilot" target="_blank">Copilot</a>, this time not from Microsoft, is a Raspberry Pi-based AI system for bicyclists that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/gadgets/2024/03/raspberry-pi-powered-ai-bike-light-detects-cars-alerts-bikers-to-bad-drivers/" target="_blank">alerts them to approaching cars</a> and cars that are driving erratically or getting too close. It’s a good example of Pete Warden’s TinyML.</li><li>Want your own <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.kleinbottle.com/" target="_blank">Klein Bottle</a>? Made by Cliff Stoll, author of the cybersecurity classic <em>The Cuckoo’s Egg</em>, who will autograph your bottle for you (and may include other surprises).</li></ul> <h2>Quantum Computing</h2> <ul><li>Google has <a href="https://bughunters.google.com/blog/5108747984306176/google-s-threat-model-for-post-quantum-cryptography" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">published</a> its threat model for quantum attacks against cryptography. The document is an excellent summary of the state of post-quantum cryptography.</li></ul> <h2>Biology</h2> <ul><li>Can <a href="https://newscenter.lbl.gov/2024/03/14/its-hearty-its-meaty-its-mold/" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">fungus</a> be engineered to produce artificial meat products? Fungus and its relatives have long been the basis of many food products, including cheese and beer. And funguses can produce the molecule that gives meat its flavor.</li></ul>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/radar-trends-to-watch-april-2024/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>ChatGPT, Author of The Quixote</title> <link>https://www.oreilly.com/radar/chatgpt-author-of-the-quixote/</link> <comments>https://www.oreilly.com/radar/chatgpt-author-of-the-quixote/#respond</comments> <pubDate>Tue, 26 Mar 2024 10:26:02 +0000</pubDate> <dc:creator><![CDATA[Hugo Bowne-Anderson]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Deep Dive]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15522</guid> <description><![CDATA[TL;DR LLMs and other GenAI models can reproduce significant chunks of training data. Specific prompts seem to “unlock” training data. We have many current and future copyright challenges: training may not infringe copyright, but legal doesn’t mean legitimate—we consider the analogy of MegaFace where surveillance models have been trained on photos of minors, for example, […]]]></description> <content:encoded><![CDATA[<h4>TL;DR</h4> <ul><li>LLMs and other GenAI models can reproduce significant chunks of training data.</li><li>Specific prompts seem to “unlock” training data.</li><li>We have many current and future copyright challenges: training may not infringe copyright, but legal doesn’t mean legitimate—we consider the analogy of MegaFace where surveillance models have been trained on photos of minors, for example, without informed consent.</li><li>Copyright was intended to incentivize cultural production: in the era of generative AI, copyright won’t be enough.</li></ul> <hr class="wp-block-separator" /> <p>In Borges’s fable “Pierre Menard, Author of The Quixote,” the eponymous Monsieur Menard plans to sit down and write a portion of Cervantes’s <em>Don Quixote</em>. Not to transcribe, but rewrite the epic novel word for word:</p> <blockquote class="wp-block-quote"><p>His goal was never the mechanical transcription of the original; he had no intention of <em>copying</em> it. His admirable ambition was to produce a number of pages which coincided—word for word and line by line—with those of Miguel de Cervantes.</p></blockquote> <p>He first tried to do so by becoming Cervantes, learning Spanish, and forgetting all the history since Cervantes wrote <em>Don Quixote</em>, among other things, but then decided it would make more sense to (re)write the text as Menard himself. The narrator tells us that “the Cervantes text and the Menard text are verbally identical, but the second is almost infinitely richer.” Perhaps this is an inversion of the ability of generative AI models (LLMs, text-to-image, and more) to reproduce swathes of their training data without those chunks being explicitly stored in the model and its weights: the output is verbally identical to the original but reproduced probabilistically without any of the human blood, sweat, tears, and life experience that goes into the creation of human writing and cultural production.</p> <h3><strong>Generative AI Has a Plagiarism Problem</strong></h3> <p>ChatGPT, for example, doesn’t <em>memorize </em>its training data per se. As Mike Loukides and Tim O’Reilly <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/copyright-ai-and-provenance/" target="_blank">astutely point out</a>:</p> <blockquote class="wp-block-quote"><p>A model prompted to write like Shakespeare may start with the word “To,” which makes it slightly more probable that it will follow that with “be,” which makes it slightly more probable that the next word will be “or”—and so forth.</p></blockquote> <p>So then, as it turns out, next-word prediction (and all the sauce on top) can reproduce chunks of training data. This is the basis of the<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.reuters.com/legal/transactional/ny-times-sues-openai-microsoft-infringing-copyrighted-work-2023-12-27/" target="_blank"><em> New York Times</em> lawsuit against OpenAI</a>. I have been able to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://chat.openai.com/c/2f11c99f-476b-446b-9b44-8cce57a20c9a" target="_blank">convince ChatGPT to give me large chunks of novels that are in the public domain</a>, such as those on Project Gutenberg, including <em>Pride and Prejudice</em>. Researchers are finding <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html" target="_blank">more and more ways</a> to extract training data from ChatGPT and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2301.13188" target="_blank">other models</a>. As far as other types of foundation models go, recent work by Gary Marcus and Reid Southern has shown that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://spectrum.ieee.org/midjourney-copyright" target="_blank">you can use Midjourney</a> (text-to-image) to generate images from <em>Star Wars</em>, <em>The Simpsons</em>, <em>Super Mario Brothers</em>, and many other films. This seems to be emerging as a feature, not a bug, and hopefully it’s obvious to you why they called their IEEE opinion piece “Generative AI Has a Visual Plagiarism Problem.” (It’s ironic that, in this article, we didn’t reproduce the images from Marcus’ article because we didn’t want to risk violating copyright—a risk that Midjourney apparently ignores and perhaps a risk that even IEEE and the authors took on!) And the space is moving quickly: <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openai.com/sora" target="_blank">Sora</a>, OpenAI’s text-to-video model, is yet to be released and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2024/02/15/1088401/openai-amazing-new-generative-ai-video-model-sora/" target="_blank">has already taken the world by storm</a>.</p> <h3><strong>Compression, Transformation, Hallucination, and Generation</strong></h3> <p><em>Training data isn’t stored in the model per se, but large chunks of it are reconstructable given the correct key (“prompt”).</em></p> <p>There are <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/information-technology/2023/09/ai-language-models-can-exceed-png-and-flac-in-lossless-compression-says-study/" target="_blank">lots of</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.perplexity.ai/search/Do-you-consider-3ffGCMwYSgKna7X5.iIdTA?s=c" target="_blank">conversations</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://vickiboykis.com/2024/01/15/whats-new-with-ml-in-production/" target="_blank">about whether</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.crosslabs.org/blog/generative-models-arent-databases" target="_blank">or</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/information-technology/2023/09/ai-language-models-can-exceed-png-and-flac-in-lossless-compression-says-study/" target="_blank">not</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://venturebeat.com/ai/llms-are-surprisingly-great-at-compressing-images-and-audio-deepmind-researchers-find/" target="_blank">LLMs</a> (and machine learning, more generally) are forms of compression or not. In many ways, they are, but they also have generative capabilities that we don’t often associate with compression.</p> <p>Ted Chiang wrote a thoughtful piece for the New Yorker called “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web" target="_blank">ChatGPT Is a Blurry JPEG of the Web</a>” that opens with the analogy of a photocopier making a slight error due to the way it compresses the digital image. It’s an interesting piece that I commend to you, but one that makes me uncomfortable. To me, the analogy breaks down before it begins: firstly, LLMs don’t merely blur, but perform highly non-linear transformations, which means you can’t just squint and get a sense of the original; secondly, for the photocopier, the error is a bug, whereas, for LLMs, all errors are features. Let me explain. Or, rather, <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://x.com/karpathy/status/1733299213503787018?s=20" target="_blank">let Andrej Karpathy explain</a>:</p> <blockquote class="wp-block-quote"><p>I always struggle a bit [when] I’m asked about the “hallucination problem” in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.</p><p>We direct their dreams with prompts. The prompts start the dream, and based on the LLM’s hazy recollection of its training documents, most of the time the result goes someplace useful.</p><p>It’s only when the dreams go into deemed factually incorrect territory that we label it a “hallucination.” It looks like a bug, but it’s just the LLM doing what it always does.</p><p>At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar “training documents” it has in its database, verbatim. You could say that this search engine has a “creativity problem”—it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.</p></blockquote> <p>As a side note, building products that strike balances between Search and LLMs will be a highly productive area and companies such as <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.perplexity.ai/" target="_blank">Perplexity AI</a> are also doing interesting work there.</p> <p>It’s interesting to me that, while LLMs are constantly “hallucinating,”<sup>1</sup> they can also reproduce large chunks of training data, not just go “someplace useful,” as Karpathy put it (summarization, for example). So, is the training data “stored” in the model? Well, no, not quite. But also… Yes? </p> <p>Let’s say I tear up a painting into a thousand pieces and put them back together in a mosaic: is the original painting stored in the mosaic? No, unless you know how to rearrange the pieces to get the original. You need a key. And, as it turns out, there happen to be certain prompts that act as keys that <em>unlock </em>training data (for insiders, you may recognize this as <a href="https://bdtechtalks.com/2021/04/23/machine-learning-membership-inference-attacks/" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">extraction attacks, a form of adversarial machine learning</a>).</p> <p>This also has implications for whether generative AI can create anything particularly novel: I have high hopes that it can, but I think that is still yet to be demonstrated. There are also significant and serious concerns about what happens when <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://pluralistic.net/2023/02/16/tweedledumber/#easily-spooked" target="_blank">we continually train models on the outputs of other models</a>.</p> <h3><strong>Implications for Copyright and Legitimacy, Big Tech, and Informed Consent</strong></h3> <p><em>Copyright isn’t the correct paradigm to be thinking about here; legal doesn’t mean legitimate; surveillance models trained on photos of your children.</em></p> <p>Now I don’t think this has implications for whether LLMs are infringing copyright and whether ChatGPT is infringing that of the<em> New York Times</em>, Sarah Silverman, George R.R. Martin, or any of us whose writing has been scraped for training data. But I also don’t think copyright is necessarily the best paradigm for thinking through whether such training and deployment should be legal or not. Firstly, copyright was created in response to the affordances of mechanical reproduction, and we now live in an age of digital reproduction, distribution, and generation. It’s also about what type of society we want to live in collectively: copyright itself was originally created to incentivize certain modes of cultural production.</p> <p>Early predecessors of modern copyright law, such as <a href="https://www.britannica.com/topic/Statute-of-Anne" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">the Statute of Anne</a> (1710) in England, were created to incentivize writers to write and to incentivize more cultural production. Up until this point, the Crown had granted exclusive rights to print certain works to the Stationers’ Company, effectively creating a monopoly, and there weren’t financial incentives to write. So, even if OpenAI and their frenemies aren’t breaching copyright law, what type of cultural production are we and aren’t we incentivizing by not zooming out and looking at as many of the externalities here as possible?</p> <p>Remember the context. Actors and writers were recently striking while <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bbc.com/news/world-us-canada-66332876" target="_blank">Netflix had an AI product manager job listing</a> with a base salary ranging from $300K to $900K USD.<sup>2</sup> Also, note that we already live in a society where many creatives end up in advertising and marketing. These may be some of the first jobs on the chopping block due to ChatGPT and friends, particularly if macroeconomic pressure keeps leaning on us all. <a href="https://openai.com/research/gpts-are-gpts" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">And that’s according to OpenAI</a>!</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/03/chatgpt-author-of-the-quixote-pic2.png" alt="" class="wp-image-15526" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/03/chatgpt-author-of-the-quixote-pic2.png 936w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/03/chatgpt-author-of-the-quixote-pic2-300x203.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/03/chatgpt-author-of-the-quixote-pic2-768x520.png 768w" sizes="(max-width: 936px) 100vw, 936px" /></figure> <p>Back to copyright: I don’t know enough about copyright law but it seems to me as though LLMs are “transformative” enough to have a fair use defense in the US. Also, <em>training</em> models doesn’t seem to me to infringe copyright because it doesn’t yet produce output! But perhaps it should infringe something: even when the collection of data is legal (which, statistically, it won’t entirely be for any web-scale corpus), it doesn’t mean it’s legitimate, and it definitely doesn’t mean there was informed consent.</p> <p>To see this, let’s consider another example, that of MegaFace. In “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.nytimes.com/interactive/2019/10/11/technology/flickr-facial-recognition.html" target="_blank">How Photos of Your Kids Are Powering Surveillance Technology</a>,” the<em> New York Times</em> reported that</p> <blockquote class="wp-block-quote"><p>One day in 2005, a mother in Evanston, Ill., joined Flickr. She uploaded some pictures of her children, Chloe and Jasper. Then she more or less forgot her account existed…<br>Years later, their faces are in a database that’s used to test and train some of the most sophisticated [facial recognition] artificial intelligence systems in the world.</p></blockquote> <p>What’s more,</p> <blockquote class="wp-block-quote"><p>Containing the likenesses of nearly 700,000 individuals, it has been downloaded by dozens of companies to train a new generation of face-identification algorithms, used to track protesters, surveil terrorists, spot problem gamblers and spy on the public at large.</p></blockquote> <p>Even in the cases where this is legal (which seem to be the vast majority of cases), it’d be tough to make an argument that it’s legitimate and even tougher to claim that there was informed consent. I also presume most people would consider it ethically dubious. I raise this example for several reasons:</p> <ul><li>Just because something is legal, doesn’t mean that we want it to be going forward.</li><li>This is illustrative of an entirely new paradigm, enabled by technology, in which vast amounts of data can be collected, processed, and used to power algorithms, models, and products; the same paradigm under which GenAI models are operating.</li><li>It’s a paradigm that’s baked into how a lot of Big Tech operates and we seem to accept it in many forms now: but if you’d built LLMs 10, let alone 20, years ago by scraping web-scale data, this would likely be a very different conversation.</li></ul> <p>I should probably also define what I mean by “legitimate/illegitimate” or at least point to a definition. When the Dutch East India Company “purchased” Manhattan from the Lenape people, Peter Minuit, who orchestrated the “purchase,” supposedly paid $24 worth of trinkets. That wasn’t illegal. Was it legitimate? It depends on your POV: not from mine. The Lenape didn’t have a conception of land ownership, just as we don’t yet have a serious conception of data ownership. This supposed “purchase” of Manhattan has resonances with uninformed consent. It’s also relevant as <a href="https://www.theguardian.com/technology/2019/jan/20/shoshana-zuboff-age-of-surveillance-capitalism-google-facebook" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">Big Tech is known for its extractive and colonialist practices</a>.</p> <h4><strong>This isn’t about copyright, the<em> New York Times</em>, or OpenAI </strong></h4> <p><em>It’s about what type of society you want to live in</em>.</p> <p>I think it’s entirely possible that the<em> New York Times</em> and OpenAI will settle out of court: OpenAI has strong incentives to do so and the <em>Times</em> likely also has short-term incentives to. However, the <em>Times</em> has also proven itself adept at playing the long game. Don’t fall into the trap of thinking this is merely about the specific case at hand. To zoom out again, we live in a society where mainstream journalism has been carved out and gutted by the internet, search, and social media. The<em> New York Times</em> is one of the last serious publications standing, and they’ve worked incredibly hard and cleverly in their “digital transformation” since the advent of the internet.<sup>3</sup></p> <p>Platforms such as Google have inserted themselves as middlemen between producers and consumers in a manner that has killed the business models of many of the content producers. They’re also disingenuous about what they’re doing: when the Australian Government was thinking of making Google pay news outlets that it linked to in Search, <a href="https://youtu.be/dHypeuHePEI?si=UJrnAJpBQeivn_sv">Google’s response was</a>:</p> <blockquote class="wp-block-quote"><p>Now remember, we don’t show full news articles, we just show you where you can go and help you to get there. Paying for links breaks the way search engines work, and it undermines how the web works, too. <strong>Let me try and say it another way. Imagine your friend asks for a coffee shop recommendation. So you tell them about a few nearby so they can choose one and go get a coffee. But then you get a bill to pay all the coffee shops, simply because you mentioned a few. </strong>When you put a price on linking to certain information, you break the way search engines work, and you no longer have a free and open web. We’re not against a new law, but we need it to be a fair one. Google has an alternative solution that supports journalism. It’s called Google News Showcase.</p></blockquote> <p>Let me be clear: Google has done incredible work in “organizing the world’s information,” but here they’re disingenuous in comparing themselves to a friend offering advice on coffee shops: friends don’t tend to have global data, AI, and infrastructural pipelines, nor are they business-predicated on surveillance capitalism.</p> <p>Copyright aside, the ability of generative AI to displace creatives is a real threat and I’m asking a real question: <em>do we want to live in a society where there aren’t many incentives for humans to write, paint, and make music?</em> Borges may not write today, given current incentives. If you don’t particularly care about Borges, perhaps you care about Philip K. Dick, Christopher Nolan, Salman Rushdie, or the Magic Realists, who were all influenced by his work.</p> <p>Beyond all the human aspects of cultural production, don’t we also still want to dream? Or do we also want to outsource that and have LLMs do all the dreaming for us?</p> <hr class="wp-block-separator" /> <h3>Footnotes</h3> <ol><li>I’m putting this in quotation marks as I’m still not entirely comfortable with the implications of anthropomorphizing LLMs in this manner.<br></li><li>My intention isn’t to suggest that Netflix is all bad. Far from it, in fact: Netflix has also been hugely powerful in providing a massive distribution channel to creatives across the globe. It’s complicated.<br></li><li>Also note that the outcome of this case could have significant impact for the future of OSS and open weight foundation models, something I hope to write about in future.</li></ol> <hr class="wp-block-separator" /> <p>This essay first appeared on <a href="https://hugobowne.github.io/hugo-blog/posts/pierre-menard/" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">Hugo Bowne-Anderson’s blog</a>. Thank you to <a href="https://www.linkedin.com/in/goku/" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">Goku Mohandas</a> for providing early feedback.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/chatgpt-author-of-the-quixote/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Vacuum Tubes and Transistors</title> <link>https://www.oreilly.com/radar/vacuum-tubes-and-transistors/</link> <comments>https://www.oreilly.com/radar/vacuum-tubes-and-transistors/#respond</comments> <pubDate>Tue, 12 Mar 2024 10:28:04 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Research]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15514</guid> <description><![CDATA[I’ve had a ham radio license since the late 1960s and observed the transition from vacuum tubes (remember them?) to transistors firsthand. Because we’re allowed to operate high-power transmitters (1,500-watt output), tubes hang on in our world a lot longer than elsewhere. There’s a good reason: tubes are ideal high-power devices for people who don’t […]]]></description> <content:encoded><![CDATA[<p>I’ve had a ham radio license since the late 1960s and observed the transition from vacuum tubes (remember them?) to transistors firsthand. Because we’re allowed to operate high-power transmitters (1,500-watt output), tubes hang on in our world a lot longer than elsewhere. There’s a good reason: tubes are ideal high-power devices for people who don’t always know what they’re doing, people who are just smart enough to be dangerous. About the only way you can damage them is by getting them hot enough to melt the internal components. That happens… but it means that there’s a huge margin for error.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/03/vacuum_article_march2024.png" alt="" class="wp-image-15515" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/03/vacuum_article_march2024.png 468w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/03/vacuum_article_march2024-226x300.png 226w" sizes="(max-width: 468px) 100vw, 468px" /><figcaption><br>3-1000Z, one of the last large glass bottle vacuum tubes. Capable of 1500W output.</figcaption></figure> <p>Transistors are the opposite. If a transistor exceeds its specifications for a millionth of a second, it will be destroyed. If tubes are like football players, transistors are like professional dancers: very strong, very powerful, but if they land wrong, there’s a serious sprain. As a result, there’s a big difference between high-power tube equipment and transistor equipment. To cool a vacuum tube, you put a fan next to it. To cool a transistor that’s generating 500 watts of heat from an area the size of a dime, you need a heavy copper spreader, a huge heat sink, and multiple fans. A tube amplifier is a box with a big power supply, a large vacuum tube, and an output circuit. A transistor amplifier has all of that, plus computers, sensors, and lots of other electronics to shut it down if anything looks like it’s going wrong. A lot of adjustments that you used to make by turning knobs have been automated. It’s easy to see the automation as a convenience, but in reality it’s a necessity. If these adjustments weren’t automated, you’d burn out the transistors before you get on the air.</p> <p>Software has been making a similar transition. The early days of the web were simple: HTML, some minimal JavaScript, CSS, and CGI. Applications have obviously been getting more complex; backends with databases, middleware, and complex frontend frameworks have all become part of our world. Attacks against applications of all kinds have grown more common and more serious. Observability is the first step in a “transistor-like” approach to building software. It’s important to make sure that you can capture enough relevant data to predict problems before they become problems; only capturing enough data for a postmortem analysis isn’t sufficient.</p> <p>Although we’re moving in the right direction, with AI the stakes are higher. This year, we’ll see AI incorporated into applications of all kinds. AI introduces many new problems that developers and IT staff will need to deal with. Here’s a start at a list:</p> <ul><li>Security issues: Whether they do it maliciously or just for lols, people will want to make your AI act incorrectly. You can expect racist, misogynist, and just plain false output. And you will find that these are business issues.</li><li>More security issues: Whether by “accident” or in response to a malicious prompt, we’ve seen that AI systems can leak users’ data to other parties.</li><li>Even more security issues: Language models are frequently used to generate source code for computer programs. That code is frequently insecure. It’s even possible that attackers could force a model to generate insecure code on their command.</li><li>Freshness: Models grow “stale” eventually and need to be retrained. There’s no evidence that large language models are an exception. Languages change slowly, but the topics about which you want your model to be conversant will not.</li><li>Copyright: While these issues are only starting to work their way through the courts, developers of AI applications will almost certainly have some liability for copyright violation.</li><li>Other liability: We’re only beginning to see legislation around privacy and transparency; Europe is the clear leader here. Whether or not the US ever has effective laws regulating the use of AI, companies need to comply with international law.</li></ul> <p>That’s <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.isc.upenn.edu/security/LLM-guide" target="_blank">only</a> a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://brightsec.com/wp-content/uploads/2023/11/Exploring-the-Risks-of-Using-Large-Language-Models-Final.pdf" target="_blank">start</a>. My point isn’t to enumerate everything that can go wrong but that complexity is growing in ways that makes in-person monitoring impossible. This is something the financial industry <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://qethanm.cc/2020/05/28/data-lessons-from-the-world-of-algorithmic-trading-part-6-monitor-your-models/" target="_blank">learned</a> a long time ago (and continues to learn). Algorithmic trading systems need to monitor themselves constantly and alert humans to intervene at the first sign something is wrong; they must have automatic “circuit breakers” to shut the application down if errors persist; and it must be possible to shut them down manually if these other methods fail. Without these safeguards, the result might look like <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Knight_Capital_Group" target="_blank">Knight Capital</a>, a company whose algorithmic trading software made <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bbc.com/news/magazine-19214294" target="_blank">$440M worth of mistakes</a> on its first day.</p> <p>The problem is that the AI industry hasn’t yet learned from the experience of others; it’s still moving fast and breaking things at the same time that it’s making the transition from relatively simple software (and yes, I consider a big React-based frontend with an enterprise backend “relatively simple” compared to LLM-based applications) to software that entangles many more processing nodes, software whose workings we don’t fully understand, and software that can cause damage at scale. And, like a modern high-power transistor amplifier, this software is too complex and fragile to be managed by hand. It’s still not clear that we know how to build the automation that we need to manage AI applications. Learning how to build those automation systems must become a priority for the next few years.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/vacuum-tubes-and-transistors/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Radar Trends to Watch: March 2024</title> <link>https://www.oreilly.com/radar/radar-trends-to-watch-march-2024/</link> <comments>https://www.oreilly.com/radar/radar-trends-to-watch-march-2024/#respond</comments> <pubDate>Tue, 05 Mar 2024 11:20:10 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[Radar Trends]]></category> <category><![CDATA[Signals]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15488</guid> <description><![CDATA[January was a dull month, at least in my opinion. Maybe everyone was recovering from their holidays. February was a short month, but it was far from dull. And I’m not even counting the first shipments of Apple Vision. OpenAI has demoed an impressive text-to-vision model called Sora; Google has two very impressive small language […]]]></description> <content:encoded><![CDATA[<p>January was a dull month, at least in my opinion. Maybe everyone was recovering from their holidays. February was a short month, but it was far from dull. And I’m not even counting the first shipments of Apple Vision. OpenAI has demoed an impressive text-to-vision model called Sora; Google has two very impressive small language models and a model specialized for time series, and they opened Gemini to the public. Outside of AI, there’s a JVM for WebAssembly; you can use it to run applications like <em>Minecraft</em> in the browser. There are some new ultralight web frameworks. And one of the world’s biggest ransomware groups has been shut down.</p> <p>On the purely weird front: there are a couple of new esoteric languages, one of which solves the problem of naming. We found out that Origami is Turing complete, so start folding. If you want a relief from AI that’s trying to be your pal, try antagonistic AI. And the best of the lot: edible robots.</p> <h2>AI</h2> <ul><li>Mistral has released <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://mistral.ai/news/mistral-large/" target="_blank">Mistral Large</a>, their flagship language model, with performance almost equal to GPT-4. It is available only via their API (although a chatbot is in beta). Unlike Mistral’s other models, Mistral Large is not open source.</li><li>This is different: Google’s DeepMind has announced <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://sites.google.com/view/genie-2024" target="_blank">Genie</a>, a generative model for building interactive worlds. It’s a video model, but unlike other video models, it’s built for game playing. (Think <em>Mario Brothers</em>, not <em>Star Wars</em>.) There are hints at other applications such as using Genie to develop virtual worlds for training other kinds of AI.</li><li>Now that large language models have been given the ability to execute other programs, they can be <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2402.06664" target="_blank">prompted to attack websites</a> and other online systems.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/vosen/ZLUDA" target="_blank">ZLUDA</a>, a library for running NVIDIA’s proprietary CUDA language on AMD GPUs, was <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.techradar.com/pro/a-lone-developer-just-open-sourced-a-tool-that-could-bring-an-end-to-nvidias-ai-hegemony-amd-financed-it-for-months-but-abruptly-ended-its-support-nobody-knows-why" target="_blank">released</a> as an open source project after AMD stopped funding it. (An earlier version targeted Intel GPUs, but that version is no longer supported.)</li><li>Researchers in China are <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2024-02-emergence-machine-language-intelligence-neural.html" target="_blank">exploring</a> whether neural networks can <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://academic.oup.com/nsr/article/11/4/nwad317/7505147?login=false" target="_blank">develop</a> their own language for images without the intermediary of human language.</li><li>The competitive programming site Topcoder has issued a challenge: <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.topcoder.com/community/innovation-challenges/intelliform-bot-gpt" target="_blank">develop an AI bot that helps people fill out government forms</a>.</li><li>Google has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://ai.google.dev/gemma" target="_blank">released</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blog.google/technology/developers/gemma-open-models/" target="_blank">two</a> small language models, Gemma 2B and Gemma 7B. They claim performance superior to Llama 2 and Mistral. The models are “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://opensource.googleblog.com/2024/02/building-open-models-responsibly-gemini-era.html" target="_blank">open</a>,” though not open source. Google has released the weights and, in addition, a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://ai.google.dev/responsible" target="_blank">responsible generative AI toolkit</a>.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://groq.com/" target="_blank">Groq</a> is a chatbot with roughly the performance of GPT-3.5 but has been tuned to give replies that are close to instantaneous.</li><li>Building an<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://wandb.ai/byyoung3/ml-news/reports/Building-a-RAG-Based-Digital-Restaurant-Menu-with-LlamaIndex-and-W-B-Weave--Vmlldzo2NjE5Njkw" target="_blank"> interactive restaurant menu with AI</a>: whether or not it’s actually useful, this is a great tutorial about building a RAG application with open source AI.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openai.com/sora" target="_blank">Sora</a> is an impressive new text-to-video model from OpenAI. It is not yet open to the public. OpenAI plans to include C2PA watermarking to identify generated video. They are currently engaged in adversarial testing to make the model less likely to generate biased or harmful content.</li><li>A research paper explores <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2402.07350" target="_blank">antagonistic AI</a>: AI that is designed to be challenging, disagreeable, and confronting. Are there applications for AI that aren’t always earnestly trying to be your friend?</li><li>The US Patent and Trademark Office has ruled that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.cnn.com/2024/02/14/tech/billions-in-ai-patents-get-new-regulations/index.html" target="_blank">only humans can patent inventions</a>, not AI. This guidance is consistent with the Copyright Office’s approach. It doesn’t mean that AI output is not patentable but that there must be significant human input directing the AI.</li><li>Google has built a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.marktechpost.com/2024/02/12/google-research-introduces-timesfm-a-single-forecasting-model-pre-trained-on-a-large-time-series-corpus-of-100b-real-world-time-points/" target="_blank">new foundation model for time series</a>. Like language models, and unlike most time series models, TimesFM is pretrained using time series data. It excels at zero-shot predictions.</li><li>OpenAI is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openai.com/blog/memory-and-new-controls-for-chatgpt" target="_blank">experimenting</a> with <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/information-technology/2024/02/amnesia-begone-soon-chatgpt-will-remember-what-you-tell-it-between-sessions/" target="_blank">long-term memory</a> in ChatGPT (i.e., memory between conversations). Long-term memory raises a number of privacy issues, in addition to more practical questions like getting a fresh start on a conversation that’s gone wrong.</li><li>AI can be an accessory in the death of traditional languages, or a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://fortune.com/2024/01/30/language-dies-ai-can-help-save-digital-extinction-demise-culture-tech/" target="_blank">tool for preserving them</a>.</li><li>There are many opportunities for using AI to improve <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://alistapart.com/article/opportunities-for-ai-in-accessibility/" target="_blank">accessibility</a>. To use AI effectively, we need to acknowledge the harm that it can do and approach accessibility issues thoughtfully.</li><li>Artificial Intelligence <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/science/2024/02/ai-cannot-be-used-to-deny-health-care-coverage-feds-clarify-to-insurers/" target="_blank">cannot be used to deny healthcare</a>. For now, at least.</li><li>Google has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blog.google/products/gemini/bard-gemini-advanced-app/" target="_blank">upgraded Bard to its latest Gemini model</a> (Gemini Advanced). It’s <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oneusefulthing.org/p/google-gemini-advanced-tasting-notes" target="_blank">worth trying</a>; it’s on a par with GPT-4V.</li><li>Hugging Face has added four new leaderboards for measuring language models’ <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/spaces/PatronusAI/enterprise_scenarios_leaderboard" target="_blank">accuracy in answering questions relevant to businesses</a> (finance, law, etc.), <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/spaces/AI-Secure/llm-trustworthy-leaderboard" target="_blank">safety and security</a>, <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/blog/leaderboards-on-the-hub-hallucinations" target="_blank">freedom from hallucinations</a>, and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/spaces/NPHardEval/NPHardEval-leaderboard" target="_blank">ability to solve reasoning problems</a>. Unfortunately, the leaderboards only evaluate open source models.</li><li>Language models can be <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/html/2401.05566v3" target="_blank">trained to be deceptive</a>—specifically, to generate code that includes security vulnerabilities given certain prompts. This behavior can be made persistent and is hard to detect and hard to remove.</li><li>Meta has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/" target="_blank">announced</a> that it will label images that have been generated with AI. They discuss a number of methods for identifying AI-generated images, including watermarking, disclosure by the creator, fact-checking, and automated classification of unmarked images.</li><li>While AI’s ability to generate music is limited, AI does an extremely good job of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/ai/2024/02/mastering-music-is-hard-can-one-click-ai-make-it-easy/" target="_blank">mastering</a> human recordings.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2024-01-team-kind-mini-ai-trillion.html" target="_blank">TinyLlama</a> is yet another new language model. TinyLlama is small: 1B parameters, but more than that, only requires 550 MB of memory to run. It was designed for small mobile and embedded devices.</li><li>The Allen Institute has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.interconnects.ai/p/olmo" target="_blank">released</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/allenai/OLMo-7B" target="_blank">OLMo</a>, an open source language model. There are 7B and 1B parameter versions, and it claims performance better than similarly sized models. OLMo is the first completely open model: every step in development and every artifact generated is available.</li><li>We have seen surprisingly little discussion of techniques for mitigating AI risks. These ideas for <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://machine-learning-made-simple.medium.com/7-methods-to-secure-llm-apps-from-prompt-injections-and-jailbreaks-11987b274012" target="_blank">protecting language models from prompt injection</a> and other attacks are far from exhaustive, but they’re a start.</li><li>Jeremy Howard has a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.youtube.com/watch?v=nOxKexn3iBo" target="_blank">video</a> on getting started with CUDA programming (NVIDIA GPU programming). It is aimed at Python programmers but no doubt useful for almost anyone.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers" target="_blank">Eagle 7B</a> is another new large language model. It claims to out-perform all 7B-class models while requiring the least computation power for inference. It is available on <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/RWKV/v5-Eagle-7B/blob/main/RWKV-v5-Eagle-World-7B-v2-20240128-ctx4096.pth" target="_blank">HuggingFace</a>. While Eagle appears to be transformer-based, it claims to point the way “beyond transformers.”</li></ul> <h2>Programming</h2> <ul><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.getstrada.com/" target="_blank">Strada</a> is a new IDE for building applications that use services from different SaaS (software as a service) providers. It makes it easier to work with multiple SaaS APIs simultaneously.</li><li>Something new for esoteric language fans: the <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/akalenuk/the_namingless_programming_language" target="_blank">namingless language</a>. Naming is hard, so this language has only one data structure (so it doesn’t need a name) and only one operator (so it doesn’t need a name, either).</li><li>Google is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://security.googleblog.com/2024/02/improving-interoperability-between-rust-and-c.html" target="_blank">supporting</a> a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://foundation.rust-lang.org/news/google-contributes-1m-to-rust-foundation-to-support-c-rust-interop-initiative/" target="_blank">Rust Foundation effort</a> to improve interoperability between C++ and Rust with the goal of enabling organizations to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/google-spends-1-million-to-make-rust-c-interoperable/" target="_blank">improve the security</a> of legacy C++ software by migrating to Rust.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/xonsh/xonsh" target="_blank">Xonsh</a> (however that may be pronounced) is a shell for Unix-like systems that combines Unix shell features with full support for Python.</li><li>Is it a coincidence? Two simple web frameworks for Java and Kotlin appear at almost the same time: <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://sparkjava.com/" target="_blank">Spark</a> and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://javalin.io/" target="_blank">Javalin</a>.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/bloomberg/memray" target="_blank">Memray</a> is a memory profiler for Python. It can track memory use in libraries written in C or C++, such as NumPy. It’s a great tool for discovering memory leaks, excessive memory allocation, and other problems.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.quantamagazine.org/how-to-build-an-origami-computer-20240130/" target="_blank">Origami is Turing complete</a>. Fold your way to solutions. Maybe we don’t need quantum computers after all.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/microsoft/microsoft-unveils-new-sudo-for-windows-feature-in-windows-11/" target="_blank">sudo on Windows</a>? The times are indeed changing. (Note that Windows sudo and Linux/WSL sudo are not the same.)</li><li>Here are some detailed <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://clig.dev/" target="_blank">guidelines</a> for designing command line user interfaces for those of us who still believe that command lines are important. They’re the only way to deal effectively with data in bulk.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.infoworld.com/article/3712706/wasm-based-cheerpj-runs-java-in-web-browsers.html" target="_blank">CheerpJ 3.0</a> is a Java Virtual Machine for WebAssembly. It is capable of running large Java applications (such as <em>Minecraft</em>) in a browser without plugins. It currently supports Java 8, but the long-term plan is to support the current long-term version (presently Java 21).</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/igor-petruk/scriptisto/wiki" target="_blank">Scriptisto</a> is a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://simonwillison.net/2024/Feb/6/scriptisto/" target="_blank">clever</a> tool that lets you write throwaway scripts in (almost) any commonly used compiled programming language. Add a simple shebang (# !/usr/bin/env/scriptso) line to any program, and it automates compilation and runs the program.</li><li>There’s yet another new language, but this one is different. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://pkl-lang.org/blog/introducing-pkl.html" target="_blank">Pkl</a> is an object-oriented language for configuration, not for general-purpose programming.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/plasma-umass/scalene" target="_blank">Scalene</a> is a new <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.itbrew.com/stories/2024/01/17/scalene-offers-efficiency-suggestions-for-python-coders" target="_blank">profiling tool for Python</a> that accounts for the difference in performance between highly optimized libraries and regular Python code. It can also ask ChatGPT for performance suggestions.</li><li>GitLab is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://docs.gitlab.com/ee/architecture/blueprints/activity_pub/" target="_blank">planning</a> to use ActivityPub (the protocol behind Mastodon and the fediverse) to connect all their Git repositories into a single network. They will start with social features, but their goal is to enable one instance to open requests for a project hosted on another instance.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://docs.docker.com/build/cloud/" target="_blank">Docker Build Cloud</a> is a service that speeds up the process of building Docker images. Claims of a 39x speedup are impressive, but even if Build Cloud doesn’t deliver quite that much, the decrease in build time is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/docker-build-cloud-promises-to-speed-docker-builds/" target="_blank">still significant</a>.</li><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.gitclear.com/coding_on_copilot_data_shows_ais_downward_pressure_on_code_quality" target="_blank">study</a> of programming trends <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://visualstudiomagazine.com/articles/2024/01/25/copilot-research.aspx" target="_blank">associates</a> the use of coding assistants like GitHub Copilot with lower-quality code, increased code churn, more copy/paste code, and less refactoring.</li></ul> <h2>Web</h2> <ul><li>Is it possible to build software with a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://digitalterroir.org/software-with-a-sense-of-place/" target="_blank">sense of place</a>? <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://digitalterroir.org/" target="_blank">Digital Terroir</a> is a fascinating discussion about what a “sense of place” might mean for digital creations.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://htmx.org/" target="_blank">htmx</a> is a lightweight JavaScript frontend library for HTML that allows web development without using JavaScript directly; rather than write JavaScript, developers add tags to standard HTML elements. Here is a good <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://semaphoreci.com/blog/htmx-react" target="_blank">comparison</a> of htmx and React.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/Kalabasa/htmz" target="_blank">htmz</a> is a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://leanrada.com/htmz/" target="_blank">minimalist</a> HTML framework that allows you to dynamically load resources within any portion of an HTML page.</li><li>The state of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://tonsky.me/blog/js-bloat/" target="_blank">JavaScript bloat</a> in 2024: it’s not pretty.</li><li>The <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://observablehq.com/blog/observable-2-0" target="_blank">Observable Framework</a> is a new static site generator for data-driven interactive web applications. It goes a step beyond notebooks, giving developers all the flexibility of modern web applications. Observable is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/observablehq/framework" target="_blank">open source</a>.</li><li>Microsoft’s Edge browser <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theregister.com/2024/01/30/microsoft_edge_tabs/" target="_blank">appears to</a> import data from the Chrome browser (tabs, stored passwords, and more) without the user’s permission, and even if the importBrowsingData setting is explicitly turned off in the user’s profile.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arc.net/max" target="_blank">Arc Max</a> is a browser that incorporates AI for summarization, asking questions of web pages, and other features. Scott Hanselman <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://hachyderm.io/@shanselman/111838693822275503" target="_blank">questions</a> whether this is a good approach.</li></ul> <h2>Security</h2> <ul><li>A new <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/new-ssh-snake-malware-steals-ssh-keys-to-spread-across-the-network/" target="_blank">attack against SSH</a> uses the SSH-Snake mapping tool to find private keys. After discovering private keys, it can easily move from one account (and machine) to another.</li><li>Law enforcement teams from several countries have <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/police-arrest-lockbit-ransomware-members-release-decryptor-in-global-crackdown/" target="_blank">arrested</a> key members of the LockBit ransomware group, seized control of its infrastructure and data, and created a free decryption tool for victims. As of February 26, though, LockBit appears to be back.</li><li>The European Court of Human Rights has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.eureporter.co/world/human-rights-category/european-court-of-human-rights-echr/2024/02/14/european-court-of-human-rights-bans-weakening-of-secure-end-to-endencryption-the-end-of-eus-chat-control-csar-mass-surveillance-plans/" target="_blank">ruled</a> that laws weakening end-to-end encryption or requiring back doors for law enforcement are illegal.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.tomshardware.com/networking/wi-fi-jamming-to-knock-out-cameras-suspected-in-nine-minnesota-burglaries-smart-security-systems-vulnerable-as-tech-becomes-cheaper-and-easier-to-acquire" target="_blank">WiFi jamming tools</a> have been used to disable security systems in a string of robberies.</li><li>A group of vulnerabilities has been discovered that allows an attacker to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/leaky-vessels-flaws-allow-hackers-to-escape-docker-runc-containers/" target="_blank">escape from a container</a>, at which point they can then access the host operating system directly.</li><li>Basic security hygiene is important. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techcrunch.com/2024/01/26/mercedez-benz-token-exposed-source-code-github/" target="_blank">An employee accidentally published</a> Mercedes-Benz’s GitHub private key in a public GitHub repository, giving anyone unlimited access to Mercedes’ source archives.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Row_hammer" target="_blank">Rowhammer</a> is an attack against a system’s memory: repeated reads and writes cause the memory to change values. A new version of the Unix/Linux <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/sudo-project/sudo/commit/7873f8334c8d31031f8cfa83bd97ac6029309e4f" target="_blank">sudo command resists rowhammer attacks</a>. It is interesting because it is a software mitigation, not a hardware fix.</li></ul> <h2>Virtual Reality</h2> <ul><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/gaming/2024/02/neurosyncs-c-infinity-tries-to-fix-vr-nausea-with-a-furniture-sized-controller/#p3" target="_blank">C-Infinity</a> is, essentially, a standing chair with built-in controllers that is designed to prevent VR-induced nausea.</li><li>Brilliant Labs is taking preorders for <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://brilliant.xyz/" target="_blank">AI glasses</a>. While there’s little description on the site, the glasses look like a heads-up augmented reality display that superimposes descriptive text on your field of view. They claim compatibility with prescription lenses.</li><li>Apple’s Vision Pro is now <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2024-02-apple-fans-vision-pro.html" target="_blank">available</a>. There are many product reviews, but <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://stratechery.com/2024/the-apple-vision-pro/" target="_blank">Ben Thompson’s review</a> is comprehensive. He identifies the big problem: apps. Not just VR apps, but AR apps, and developing that new generation of apps may require investments that few companies can afford.</li></ul> <h2>Biology</h2> <ul><li>Several years ago, a Kickstarter project to create a glow-in-the-dark rose failed. Now you can order a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2024/02/22/1088768/glowing-plant-lightbio-purple-tomato-norfolk/" target="_blank">glowing petunia</a> online, along with purple tomatoes. Has synthetic biology arrived?</li><li><a href="https://techxplore.com/news/2024-02-edible-electronics-hospital-patients.html" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">Robots you can eat</a>: Researchers are designing robots, including electronics and actuators, that are entirely edible: honey can possibly act as a semiconductor, gold leaf can be used as wire, and batteries can be made from food materials. There may be applications in medicine.</li></ul> <h2>Energy</h2> <ul><li>An abandoned Finnish copper mine will be <a href="https://www.independent.co.uk/tech/gravity-battery-mine-renewable-energy-b2492087.html" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">repurposed</a> as a giant gravity battery that can store excess energy from renewable sources. It isn’t clear how long the battery can run before “discharging” or what the total energy storage is.</li></ul>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/radar-trends-to-watch-march-2024/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Universal API Access from Postgres and SQLite</title> <link>https://www.oreilly.com/radar/universal-api-access-from-postgres-and-sqlite/</link> <comments>https://www.oreilly.com/radar/universal-api-access-from-postgres-and-sqlite/#respond</comments> <pubDate>Tue, 27 Feb 2024 13:39:58 +0000</pubDate> <dc:creator><![CDATA[Jon Udell]]></dc:creator> <category><![CDATA[Data]]></category> <category><![CDATA[Radar Column]]></category> <category><![CDATA[Research]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15473</guid> <description><![CDATA[In “SQL: The Universal Solvent for REST APIs” we saw how Steampipe’s suite of open source plug-ins that translate REST API calls directly into SQL tables. These plug-ins were, until recently, tightly bound to the open source engine and to the instance of Postgres that it launches and controls. That led members of the Steampipe community to ask: “Can […]]]></description> <content:encoded><![CDATA[<p>In “<a rel="noreferrer noopener" href="https://www.oreilly.com/radar/sql-the-universal-solvent-for-rest-apis/" target="_blank">SQL: The Universal Solvent for REST APIs</a>” we saw how Steampipe’s suite of open source plug-ins that translate REST API calls directly into SQL tables. These plug-ins were, until recently, tightly bound to the <a rel="noreferrer noopener" href="https://github.com/turbot/steampipe" target="_blank">open source engine</a> and to the instance of Postgres that it launches and controls. That led members of the Steampipe community to ask: “Can we use the plug-ins in our own Postgres databases?” Now the answer is yes—and more—but let’s focus on Postgres first.</p> <div class="wp-block-group has-very-light-gray-background-color has-background"><div class="wp-block-group__inner-container"><div class="wp-block-group"><div class="wp-block-group__inner-container"><p><strong>NOTE:</strong> Each Steampipe plugin ecosystem is now also a standalone foreign-data-wrapper extension for Postgres, a virtual-table extension for SQLite, and an export tool.</p></div></div></div></div> <h2>Using a Steampipe Plugin as a Standalone Postgres Foreign Data Wrapper (FDW)</h2> <p>Visit <a rel="noreferrer noopener" href="https://steampipe.io/downloads?install=postgres" target="_blank">Steampipe downloads</a> to find the installer for your OS, and run it to acquire the Postgres FDW distribution of a plugin—in this case, <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/github" target="_blank">the GitHub plugin</a>. It’s one of (currently) 140 plug-ins available on the <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins" target="_blank">Steampipe hub</a>. Each plugin provides a set of tables that map API calls to database tables—in the case of the GitHub plugin, <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/github/tables" target="_blank">55 such tables</a>. Each table can appear in a <code>FROM</code> or <code>JOIN</code> clause; here’s a query to select columns from the <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/github/tables" target="_blank">GitHub</a> issue, filtering on a repository and author.</p> <pre class="wp-block-code"><code>select state, updated_at, title, urlfrom github_issuewhere repository_full_name = 'turbot/steampipe' and author_login = 'judell'order by updated_at desc</code></pre> <p>If you’re using Steampipe, you can install the GitHub plugin like this:</p> <pre class="wp-block-code"><code>steampipe plugin install github</code></pre> <p>then run the query in the Steampipe CLI or in any Postgres client that can connect to Steampipe’s instance of Postgres.</p> <p>But if you want to do the same thing in your own instance of Postgres, you can install the plugin in a different way.</p> <pre class="wp-block-code"><code>$ sudo /bin/sh -c "$( curl -fsSL https://steampipe.io/install/postgres.sh)"Enter the plugin name: githubEnter the version (latest): Discovered:- PostgreSQL version: 14- PostgreSQL location: /usr/lib/postgresql/14- Operating system: Linux- System architecture: x86_64 Based on the above, steampipe_postgres_github.pg14.linux_amd64.tar.gzwill be downloaded, extracted and installed at: /usr/lib/postgresql/14 Proceed with installing Steampipe PostgreSQL FDW for version 14 at /usr/lib/postgresql/14?- Press 'y' to continue with the current version.- Press 'n' to customize your PostgreSQL installation directory and select a different version. (Y/n): Downloading steampipe_postgres_github.pg14.linux_amd64.tar.gz...########################################################################################### 100.0%steampipe_postgres_github.pg14.linux_amd64/steampipe_postgres_github.pg14.linux_amd64/steampipe_postgres_github.sosteampipe_postgres_github.pg14.linux_amd64/steampipe_postgres_github.controlsteampipe_postgres_github.pg14.linux_amd64/steampipe_postgres_github--1.0.sqlsteampipe_postgres_github.pg14.linux_amd64/install.shsteampipe_postgres_github.pg14.linux_amd64/README.md Download and extraction completed. Installing steampipe_postgres_github in /usr/lib/postgresql/14... Successfully installed steampipe_postgres_github extension! Files have been copied to:- Library directory: /usr/lib/postgresql/14/lib- Extension directory: /usr/share/postgresql/14/extension/</code></pre> <p>Now connect to your server as usual, using <code>psql</code> or another client, most typically as the <code>postgres</code> user. Then run these commands, which are typical for any Postgres <a rel="noreferrer noopener" href="https://wiki.postgresql.org/wiki/Foreign_data_wrappers" target="_blank">foreign data wrapper</a>. As with all Postgres extensions, you start like this:</p> <pre class="wp-block-code"><code>CREATE EXTENSION steampipe_postgres_fdw_github;</code></pre> <p>To use a foreign data wrapper, you first create a server:</p> <pre class="wp-block-code"><code>CREATE SERVER steampipe_github FOREIGN DATA WRAPPERsteampipe_postgres_github OPTIONS (config 'token="ghp_..."');</code></pre> <p>Use OPTIONS to configure the extension to use your GitHub access token. (Alternatively, the standard environment variables used to configure a Steampipe plugin—it’s just <code>GITHUB_TOKEN</code> in this case—will work if you set them before starting your instance of Postgres.)</p> <p>The <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/github/tables" target="_blank">tables</a> provided by the extension will live in a schema, so define one:</p> <pre class="wp-block-code"><code>CREATE SCHEMA github;</code></pre> <p>Now import the schema defined by the foreign server into the local schema you just created:</p> <pre class="wp-block-code"><code>IMPORT FOREIGN SCHEMA github FROM SERVER steampipe_github INTO github;</code></pre> <p>Now run a query!</p> <p>The foreign tables provided by the extension live in the <code>github schema</code>, so by default you’ll refer to tables like <code>github.github_my_repository</code>. If you <code>set search_path = 'github'</code>, though, the schema becomes optional and you can write queries using unqualified table names. Here’s a query we showed last time. It uses the <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/github/tables/github_search_repository" target="_blank">GitHub_search_repository</a> which encapsulates the GitHub API for <a rel="noreferrer noopener" href="https://docs.github.com/en/search-github/searching-on-github/searching-for-repositories" target="_blank">searching repositories</a>.</p> <p>Suppose you’re looking for repos related to PySpark. Here’s a query to find repos whose names match “pyspark” and report a few metrics to help you gauge activity and popularity.</p> <pre class="wp-block-code"><code>select name_with_owner, updated_at, -- how recently updated? stargazer_count -- how many people starred the repo?from github_search_repository where query = 'pyspark in:name' order by stargazer_count desclimit 10;+---------------------------------------+------------+---------------+|name_with_owner |updated_at |stargazer_count|+---------------------------------------+------------+---------------+| AlexIoannides/pyspark-example-project | 2024-02-09 | 1324 || mahmoudparsian/pyspark-tutorial | 2024-02-11 | 1077 || spark-examples/pyspark-examples | 2024-02-11 | 1007 || palantir/pyspark-style-guide | 2024-02-12 | 924 || pyspark-ai/pyspark-ai | 2024-02-12 | 791 || lyhue1991/eat_pyspark_in_10_days | 2024-02-01 | 719 || UrbanInstitute/pyspark-tutorials | 2024-01-21 | 400 || krishnaik06/Pyspark-With-Python | 2024-02-11 | 400 || ekampf/PySpark-Boilerplate | 2024-02-11 | 388 || commoncrawl/cc-pyspark | 2024-02-12 | 361 |+---------------------------------------+------------+---------------+</code></pre> <p>If you have a lot of repos, the first run of that query will take a few seconds. The second run will return results instantly, though, because the extension includes a powerful and sophisticated <a rel="noreferrer noopener" href="https://steampipe.io/blog/release-0-10-0#caching-row-subsets" target="_blank">cache</a>.</p> <p>And that’s all there is to it! Every Steampipe plugin is now also a foreign data wrapper that works exactly like this one. You can load multiple extensions in order to <a rel="noreferrer noopener" href="https://steampipe.io/blog/use-shodan-to-test-aws-public-ip" target="_blank">join across APIs</a>. Of course, you can join any of these API-sourced foreign tables with your own Postgres tables. And to save the results of any query, you can prepend “create table NAME as” or “create materialized view NAME as” to a query to persist results as a table or view.</p> <h2>Using a Steampipe Plugin as a SQLite Extension That Provides Virtual Tables</h2> <p>Visit <a rel="noreferrer noopener" href="https://steampipe.io/downloads?install=postgres" target="_blank">Steampipe downloads</a> to find the installer for your OS and run it to acquire the SQLite distribution of the same plugin.</p> <pre class="wp-block-code"><code>$ sudo /bin/sh -c "$(curl -fsSL https://steampipe.io/install/sqlite.sh)"Enter the plugin name: githubEnter version (latest): Enter location (current directory): Downloading steampipe_sqlite_github.linux_amd64.tar.gz...############################################################################ 100.0%steampipe_sqlite_github.so steampipe_sqlite_github.linux_amd64.tar.gz downloaded and extracted successfully at /home/jon/steampipe-sqlite.</code></pre> <p>Here’s the setup, and you can place this code in <code>~/.sqliterc</code> if you want to run it every time you start <code>sqlite</code>.</p> <pre class="wp-block-code"><code>.load /home/jon/steampipe-sqlite/steampipe_sqlite_github.so select steampipe_configure_github(' token="ghp_..."');</code></pre> <p>Now you can run the same query as above. Here, too, the results are cached, so a second run of the query will be instant.</p> <p>What about the differences between Postgres-flavored and SQLite-flavored SQL? The <a rel="noreferrer noopener" href="https://hub.steampipe.io/" target="_blank">Steampipe hub</a> is your friend! For example, here are Postgres and SQLite variants of a query that accesses a field inside a JSON column in order to tabulate the languages associated with your gists.</p> <h3>Postgres</h3> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/pg-gist-summary.png" alt="" class="wp-image-15451" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/pg-gist-summary.png 776w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/pg-gist-summary-300x145.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/pg-gist-summary-768x370.png 768w" sizes="(max-width: 776px) 100vw, 776px" /></figure> <h3>SQLite</h3> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/sqlite-gist-summary.png" alt="" class="wp-image-15452" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/sqlite-gist-summary.png 777w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/sqlite-gist-summary-300x146.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/02/sqlite-gist-summary-768x374.png 768w" sizes="(max-width: 777px) 100vw, 777px" /></figure> <p>The <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/github/tables/github_my_gist" target="_blank">github_my_gist</a> table reports <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/github/tables/github_my_gist#inspect" target="_blank">details</a> about gists that belong to the GitHub user who is authenticated to Steampipe. The language associated with each gist lives in a JSONB column called files, which contains a list of objects like this.</p> <pre class="wp-block-code"><code>{ "size": 24541, "type": "text/markdown", "raw_url": "https://gist.githubusercontent.com/judell/49d66ca2a5d2a3b "filename": "steampipe-readme-update.md", "language": "Markdown"}</code></pre> <p>The functions needed to project that list as rows differ: in Postgres you use <code>jsonb_array_elements</code> and in SQLite it’s <code>json_each</code>.</p> <p>As with Postgres extensions, you can load multiple SQLite extensions in order to <a rel="noreferrer noopener" href="https://steampipe.io/blog/use-shodan-to-test-aws-public-ip" target="_blank">join across APIs</a>. You can join any of these API-sourced foreign tables with your own SQLite tables. And you can prepend <code>create table NAME as</code> to a query to persist results as a table.</p> <h2>Using a Steampipe Plugin as a Standalone Export Tool</h2> <p>Visit <a rel="noreferrer noopener" href="https://steampipe.io/downloads?install=export" target="_blank">Steampipe downloads</a> to find the installer for your OS, and run it to acquire the export distribution of a plugin—again, we’ll illustrate using <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/github" target="_blank">the GitHub plugin</a>.</p> <pre class="wp-block-code"><code>$ sudo /bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)"Enter the plugin name: githubEnter the version (latest): Enter location (/usr/local/bin): Created temporary directory at /tmp/tmp.48QsUo6CLF. Downloading steampipe_export_github.linux_amd64.tar.gz...############################################################################## 100.0%Deflating downloaded archivesteampipe_export_githubInstallingApplying necessary permissionsRemoving downloaded archivesteampipe_export_github was installed successfully to/usr/local/bin$ steampipe_export_github -hExport data using the github plugin. Find detailed usage information including table names, column names, and examples at the Steampipe Hub:https://hub.steampipe.io/plugins/turbot/github Usage: steampipe_export_github TABLE_NAME [flags] Flags: --config string Config file data -h, --help help for steampipe_export_github --limit int Limit data --output string Output format: csv, json or jsonl (default "csv") --select strings Column data to display --where stringArray where clause data</code></pre> <p>There’s no SQL engine in the picture here; this tool is purely an exporter. To export all your gists to a JSON file:</p> <pre class="wp-block-code"><code>steampipe_export_github github_my_gist --output json > gists.json</code></pre> <p>To select only some columns and export to a CSV file:</p> <pre class="wp-block-code"><code>steampipe_export_github github_my_gist --output csv --select "description,created_at,html_url" > gists.csv</code></pre> <p>You can use <code>--limit</code> to limit the rows returned and <code>--where</code> to filter them, but mostly you’ll use this tool to quickly and easily grab data that you’ll massage elsewhere, for example, in a spreadsheet.</p> <h2>Tap into the Steampipe Plugin Ecosystem</h2> <p>Steampipe plug-ins aren’t just raw interfaces to underlying APIs. They use tables to model those APIs in useful ways. For example, the <code>github_my_repository</code> table exemplifies a design pattern that applies consistently across the suite of plug-ins. From the GitHub plugin’s documentation:</p> <blockquote class="wp-block-quote"><p>You can own repositories individually, or you can share ownership of repositories with other people in an organization. The <code>github_my_repository</code> table will list repos that you own, that you collaborate on, or that belong to your organizations. To query ANY repository, including public repos, use the <code>github_repository</code> table.</p></blockquote> <p>Other plug-ins follow the same pattern. For example, the <a href="https://hub.steampipe.io/plugins/turbot/microsoft365">Microsoft 365</a> plugin provides both <code>microsoft_my_mail_message</code> and <code>microsoft_mail_message</code>, and the plugin provides <code>googleworkspace_my_gmail_message</code> and <code>googleworkspace_gmail</code>. Where possible, plug-ins consolidate views of resources from the perspective of an authenticated user.</p> <p>While plug-ins typically provide tables with fixed schemas, that’s not always the case. Dynamic schemas, implemented by the <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/francois2metz/airtable" target="_blank">Airtable</a>, <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/csv" target="_blank">CSV</a>, <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/kubernetes" target="_blank">Kubernetes</a>, and <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/salesforce" target="_blank">Salesforce</a> plug-ins (among others) are another key pattern. Here’s a CSV example using a standalone Postgres FDW.</p> <pre class="wp-block-code"><code>IMPORT FOREIGN SCHEMA csv FROM SERVER steampipe_csv INTO csv OPTIONS(config 'paths=["/home/jon/csv"]');</code></pre> <p>Now all the <code>.csv</code> files in <code>/home/jon/csv</code> will automagically be Postgres foreign tables. Suppose you keep track of valid owners of EC2 instances in a file called <code>ec2_owner_tags</code>. Here’s a query against the corresponding table.</p> <pre class="wp-block-code"><code>select * from csv.ec2_owner_tags; owner | _ctx----------------+---------------------------- Pam Beesly | {"connection_name": "csv"} Dwight Schrute | {"connection_name": "csv"}</code></pre> <p>You could join that table with the <a rel="noreferrer noopener" href="https://hub.steampipe.io/plugins/turbot/aws" target="_blank">AWS plugin’s</a> <code>aws_ec2_instance</code> table to report owner tags on EC2 instances that are (or are not) listed in the CSV file.</p> <pre class="wp-block-code"><code>select ec2.owner, case when csv.owner is null then 'false' else 'true' end as is_listedfrom (select distinct tags ->> 'owner' as owner from aws.aws_ec2_instance) ec2left join csv.ec2_owner_tags csv on ec2.owner = csv.owner; owner | is_listed----------------+----------- Dwight Schrute | true Michael Scott | false</code></pre> <p>Across the suite of plug-ins there are more than 2,300 predefined fixed-schema tables that you can use in these ways, plus an unlimited number of dynamic tables. And new plug-ins are constantly being added by Turbot and by Steampipe’s open source community. You can tap into this ecosystem using <a rel="noreferrer noopener" href="https://steampipe.io/" target="_blank">Steampipe</a> or <a rel="noreferrer noopener" href="https://pipes.turbot.com/" target="_blank">Turbot Pipes</a>, from your own Postgres or SQLite database, or directly from the command line.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/universal-api-access-from-postgres-and-sqlite/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Corporate Responsibility in the Age of AI</title> <link>https://www.oreilly.com/radar/corporate-responsibility-in-the-age-of-ai/</link> <comments>https://www.oreilly.com/radar/corporate-responsibility-in-the-age-of-ai/#respond</comments> <pubDate>Tue, 20 Feb 2024 17:02:29 +0000</pubDate> <dc:creator><![CDATA[Laura Baldwin and Mike Loukides]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Business]]></category> <category><![CDATA[Deep Dive]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15409</guid> <description><![CDATA[Since its release in November 2022, almost everyone involved with technology has experimented with ChatGPT: students, faculty, and professionals in almost every discipline. Almost every company has undertaken AI projects, including companies that, at least on the face of it, have “no AI” policies. Last August, OpenAI stated that 80% of Fortune 500 companies have […]]]></description> <content:encoded><![CDATA[<p>Since its release in November 2022, almost everyone involved with technology has experimented with ChatGPT: students, faculty, and professionals in almost every discipline. Almost every company has undertaken AI projects, including companies that, at least on the face of it, have “no AI” policies. Last August, OpenAI <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openai.com/blog/introducing-chatgpt-enterprise" target="_blank">stated</a> that 80% of Fortune 500 companies have ChatGPT accounts. Interest and usage have increased as OpenAI has released more capable versions of its language model: GPT-3.5 led to GPT-4 and multimodal GPT-4V, and OpenAI has announced an Enterprise service with better guarantees for security and privacy. Google’s Bard/Gemini, Anthropic’s Claude, and other models have made similar improvements. AI is everywhere, and even if the initial frenzy around ChatGPT has died down, the big picture hardly changes. If it’s not ChatGPT, it will be something else, possibly something users aren’t even aware of: AI tools embedded in documents, spreadsheets, slide decks, and other tools in which AI fades into the background. AI will become part of almost every job, ranging from manual labor to management.</p> <p>With that in mind, we need to ask what companies must do to use AI responsibly. Ethical obligations and responsibilities don’t change, and we shouldn’t expect them to. The problem that AI introduces is the scale at which automated systems can cause harm. AI magnifies issues that are easily rectified when they affect a single person. For example, every company makes poor hiring decisions from time to time, but with AI all your hiring decisions can quickly become questionable, as Amazon <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.reuters.com/article/idUSKCN1MK0AG/" target="_blank">discovered</a>. The <em>New York Times</em>’ lawsuit against OpenAI isn’t about a single article; if it were, it would hardly be worth the legal fees. It’s about scale, the potential for reproducing its whole archive. O’Reilly has built an AI application that uses our authors’ content to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/answers/search/" target="_blank">answer questions</a>, but we compensate our authors fairly for that use: we won’t ignore our obligations to our authors, either individually or at scale.</p> <p>It’s essential for companies to come to grips with the scale at which AI works and the effects it creates. What are a corporation’s responsibilities in the age of AI—to its employees, its customers, and its shareholders? The answers to this question will define the next generation of our economy. Introducing new technology like AI doesn’t change a company’s basic responsibilities. However, companies must be careful to continue living up to their responsibilities. Workers fear losing their jobs “to AI” but also look forward to tools that can eliminate boring, repetitive tasks. Customers fear even worse interactions with customer service but look forward to new kinds of products. Stockholders anticipate higher profit margins but fear seeing their investments evaporate if companies can’t adopt AI quickly enough. Does everybody win? How do you balance the hopes against the fears? Many people believe that a corporation’s sole responsibility is to maximize short-term shareholder value with little or no concern for the long term. In that scenario, everybody loses—including stockholders who don’t realize they’re participating in a scam.</p> <p>How would corporations behave if their goal were to make life better for all of their stakeholders? That question is inherently about scale. Historically, the stakeholders in any company are the stockholders. We need to go beyond that: the employees are also stakeholders, as are the customers, as are the business partners, as are the neighbors and, in the broadest sense, anyone participating in the economy. We need a balanced approach to the entire ecosystem.</p> <p>O’Reilly tries to operate in a balanced ecosystem with equal weight going toward customers, shareholders, and employees. We’ve made a conscious decision not to manage our company for the good of one group while disregarding the needs of everyone else. From that perspective, we want to dive into how we believe companies need to think about AI adoption and how their implementation of AI needs to work for the benefit of all three constituencies.</p> <h2>Being a Responsible Employer</h2> <p>While the number of jobs lost to AI so far has been small, it’s not zero. Several <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.itechpost.com/articles/117796/20230605/content-writer-loses-clients-chatgpt-turns-trade-work.htm" target="_blank">copywriters</a> have reported being replaced by ChatGPT; one of them eventually <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.dailydot.com/news/laid-off-worker-interviews-ai-job/" target="_blank">had to</a> “accept a position training AI to do her old job.” However, a few copywriters don’t make a trend. So far, the total numbers appear to be small. One report claims that in May 2023, over <a href="https://gizmodo.com/jobs-ai-ai-jobs-chatgpt-layoffs-1850505874">80,000</a> workers were laid off, but only about 4,000 of these layoffs were caused by AI, or 5%. That’s a very partial picture of an economy that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bls.gov/news.release/archives/empsit_06022023.htm" target="_blank">added 390,000 jobs</a> during the same period. But before dismissing the fear-mongering, we should wonder whether this is the shape of things to come. 4,000 layoffs could become a much larger number very quickly.</p> <p>Fear of losing jobs to AI is probably lower in the technology sector than in other business sectors. Programmers have always made tools to make their jobs easier, and GitHub Copilot, the GPT family of models, Google’s Gemini, and other language models are tools that they’re already taking advantage of. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/fearing-the-wrong-thing/" target="_blank">For the immediate future</a>, productivity improvements are likely to be relatively small: 20% at most. However, that doesn’t negate the fear; and there may well be more fear in other sectors of the economy. Truckers and taxi drivers wonder about autonomous vehicles; writers (including novelists and screenwriters, in addition to marketing copywriters) worry about text generation; customer service personnel worry about chatbots; teachers worry about automated tutors; and managers worry about tools for creating strategies, automating reviews, and much more.</p> <p>An easy <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://hbr.org/2023/08/ai-wont-replace-humans-but-humans-with-ai-will-replace-humans-without-ai" target="_blank">reply</a> to all this fear is “AI is not going to replace humans, but humans with AI are going to replace humans without AI.” We agree with that statement, as far as it goes. But it doesn’t go very far. This attitude blames the victim: if you lose your job, it’s your own fault for not learning how to use AI. That’s a gross oversimplification. Second, while most technological changes have created more jobs than they destroyed, that doesn’t mean that there isn’t a time of dislocation, a time when the old professions are dying out but the new ones haven’t yet come into being. We believe that AI will create more jobs than it destroys—but what about that transition period? The World Economic Forum has published a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.weforum.org/agenda/2023/05/jobs-lost-created-ai-gpt/" target="_blank">short report</a> that lists the 10 jobs most likely to see a decline, and the 10 most likely to see gains. Suffice it to say that if your job title includes the word “clerk,” things might not look good—but your prospects are looking up if your job title includes the word “engineer” or “analyst.”</p> <p>The best way for a company to honor its commitment to its employees and to prepare for the future is through education. Most jobs won’t disappear, but all jobs will change. Providing appropriate training to get employees through that change may be a company’s biggest responsibility. Learning how to use AI effectively isn’t as trivial as a few minutes of playing with ChatGPT makes it appear. Developing good prompts is serious work and it requires training. That’s certainly true for technical employees who will be developing applications that use AI systems through an API. It’s also true for nontechnical employees who may be trying to find insights from data in a spreadsheet, summarize a group of documents, or write text for a company report. AI needs to be told exactly what to do and, often, how to do it.</p> <p>One aspect of this change will be verifying that the output of an AI system is correct. Everyone knows that language models make mistakes, often called “hallucinations.” While these inaccuracies may not be as dramatic as <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.reuters.com/legal/new-york-lawyers-sanctioned-using-fake-chatgpt-cases-legal-brief-2023-06-22/" target="_blank">making up case law</a>, AI will make mistakes—errors at the scale of AI—and users will need to know how to check its output without being deceived (or in some cases, bullied) by its overconfident voice. The frequency of errors may go down as AI technology improves, but errors won’t disappear in the foreseeable future. And even with error rates as low as 1%, we’re easily talking about thousands of errors sprinkled randomly through software, press releases, hiring decisions, catalog entries—everything AI touches. In many cases, verifying that an AI has done its work correctly may be as difficult as it would be for a human to do the work in the first place. This process is often called “critical thinking,” but it goes a lot deeper: it requires scrutinizing every fact and every logical inference, even the most self-evident and obvious. There is a methodology that needs to be taught, and it is the employers’ responsibility to ensure that their employees have appropriate training to detect and correct errors.</p> <p>The responsibility for education isn’t limited to training employees to use AI within their current positions. Companies need to provide education for transitions from jobs that are disappearing to jobs that are growing. Responsible use of AI includes auditing to ensure that its outputs aren’t biased, and that they are appropriate. Customer service personnel can be retrained to test and verify that AI systems are working correctly. Accountants can become auditors responsible for overseeing IT security. That transition is already happening; auditing for the SOC 2 corporate security certification is handled by <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.checkpoint.com/cyber-hub/cyber-security/what-is-soc-2-compliance/" target="_blank">accountants</a>. Businesses need to invest in training to support transitions like these.</p> <p>Looking at an even broader context: what are a corporation’s responsibilities to local public education? No company is going to prosper if it can’t hire the people it needs. And while a company can always hire employees who aren’t local, that assumes that educational systems across the country are well-funded, but they frequently aren’t.</p> <p>This looks like a “tragedy of the commons”: no single nongovernmental organization is responsible for the state of public education, and public education is expensive (it’s usually the biggest line item on any municipal budget), so nobody takes care of it. But that narrative repeats a fundamental misunderstanding of the “commons.” The “tragedy of the commons” narrative was never correct; it is a fiction that achieved prominence as an <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blogs.scientificamerican.com/voices/the-tragedy-of-the-tragedy-of-the-commons/" target="_blank">argument to justify eugenics</a> and other racist policies. Historically, common lands were <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://dlc.dlib.indiana.edu/dlc/bitstream/handle/10535/3113/buck_NoTragedy.pdf?sequence=1&isAllowed=y" target="_blank">well managed by law, custom</a>, and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.cambridge.org/us/universitypress/subjects/politics-international-relations/political-theory/governing-commons-evolution-institutions-collective-action-1?format=PB&isbn=9781107569782" target="_blank">voluntary associations</a>. The commons declined when landed gentry and other large landholders abused their rights to the detriment of the small farmers; the commons as such disappeared through enclosure, when the large landholders fenced in and claimed common land as private property. In the context of the 20th and 21st centuries, the landed gentry—now frequently multinational corporations—protect their stock prices by negotiating tax exemptions and abandoning their responsibilities toward their neighbors and their employees.</p> <p>The economy itself is the biggest commons of all, and nostrums like “the invisible hand of the marketplace” do little to help us understand responsibilities. This is where the modern version of “enclosure” takes place: in minimizing labor cost to maximize short-term value and executive salaries. In a winner-take-all economy where a company’s highest-paid employees can earn over <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theguardian.com/us-news/2022/jun/07/us-wage-gap-ceos-workers-institute-for-policy-studies-report" target="_blank">1,000 times</a> as much as the lowest paid, the absence of a commitment to employees leads to poor housing, poor school systems, poor infrastructure, and marginalized local businesses. Quoting <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://knarf.english.upenn.edu/Smith/tms411.html" target="_blank">a line from Adam Smith</a> that hasn’t entered our set of economic cliches, senior management salaries shouldn’t facilitate “gratification of their own vain and insatiable desires.”</p> <p>One part of a company’s responsibilities to its employees is paying a fair wage. The consequences of not paying a fair wage, or of taking every opportunity to minimize staff, are far-reaching; they aren’t limited to the people who are directly affected. When employees aren’t paid well or live in fear of layoffs, they can’t participate in the local economy. There’s a reason that low-income areas often don’t have basic services like banks or supermarkets. When people are just subsisting, they can’t afford the services they need to flourish; they live on junk food because they can’t afford a $40 Uber to the supermarket in a more affluent town (to say nothing of the time). And there’s a reason why it’s difficult for lower-income people to make the transition to the middle class. In very real terms, living is more expensive if you’re poor: long commutes with less reliable transportation, poor access to healthcare, more expensive food, and even higher rents (slum apartments aren’t cheap) make it very difficult to escape poverty. An automobile repair or a doctor’s bill can exhaust the savings of someone who is near the poverty line.</p> <p>That’s a local problem, but it can compound into a national or worldwide problem. That happens when layoffs become widespread—as happened in the winter and spring of 2023. Although there was little evidence of economic stress, fear of a recession led to widespread layoffs (often sparked by “activist investors” seeking only to maximize short-term stock price), which nearly caused a real recession. The primary driver for this “media recession” was a vicious cycle of layoff news, which encouraged fear, which led to more layoffs. When you see weekly announcements of layoffs in the tens of thousands, it’s easy to follow the trend. And that trend will eventually lead to a downward spiral: people who are unemployed don’t go to restaurants, defer maintenance on cars and houses, spend less on clothing, and economize in many other ways. Eventually, this reduction in economic activity trickles down and causes merchants and other businesses to close or reduce staff.</p> <p>There are times when layoffs are necessary; O’Reilly has suffered through those. We’re still here as a result. Changes in markets, corporate structure, corporate priorities, skills required, and even strategic errors such as overhiring can all make layoffs necessary. These are all valid reasons for layoffs. A layoff should never be an “All of our peers are laying people off, let’s join the party” event; that happened all too often in the technology sector last year. Nor should it be an “our stock price could be higher and the board is cranky” event. A related responsibility is honesty about the company’s economic condition. Few employees will be surprised to hear that their company isn’t meeting its financial goals. But honesty about what everyone already knows might keep key people from leaving when you can least afford it. Employees who haven’t been treated with respect and honesty can’t be expected to show loyalty when there’s a crisis.</p> <p>Employers are also responsible for healthcare, at least in the US. This is hardly ideal, but it’s not likely to change in the near future. Without insurance, a hospitalization can be a financial disaster, even for a highly compensated employee. So can a cancer diagnosis or any number of chronic diseases. Sick time is another aspect of healthcare—not just for those who are sick, but for those who work in an office. The COVID pandemic is “over” (for a very limited sense of “over”) and many companies are asking their staff to return to offices. But we all know people who at workplaces where COVID, the flu, or another disease has spread like wildfire because one person didn’t feel well and reported to the office anyway. Companies need to respect their employees’ health by providing health insurance and allowing sick time—both for the employees’ sakes and for everyone they come in contact with at work.</p> <p>We’ve gone far afield from AI but for good reasons. A new technology can reveal gaps in corporate responsibility and help us think about what those responsibilities should be. Compartmentalizing is unhealthy; it’s not helpful to talk about a company’s responsibilities to highly paid engineers developing AI systems without connecting that to responsibilities toward the lowest-paid support staff. If programmers are concerned about being replaced by a generative algorithm, the groundskeepers should certainly worry about being replaced by autonomous lawnmowers.</p> <p>Given this context, what are a company’s responsibilities to all of its employees?</p> <ul><li>Providing training for employees so they remain relevant even as their jobs change</li><li>Providing insurance and sick leave so that employees’ livelihoods aren’t threatened by health problems</li><li>Paying a livable wage that allows employees and the communities they live in to prosper</li><li>Being honest about the company’s finances when layoffs or restructuring are likely</li><li>Balancing the company’s responsibilities to employees, customers, investors, and other constituencies</li></ul> <h2>Responsibilities to Business Partners</h2> <p>Generative AI has spawned a swirl of controversy around copyright and intellectual property. Does a company have any obligation toward the creators of content that they use to train their systems? These content creators are business partners, whether or not they have any say in the matter. A company’s legal obligations are currently unclear and will ultimately be decided in the courts or by legislation. But treating its business partners fairly and responsibly isn’t just a legal matter.</p> <p>We believe that our talent—authors and teachers—should be paid. As a company that is using AI to generate and deliver content, we are committed to allocating income to authors as their work is used in that content and paying them appropriately—as we do with all other media. Granted, our use case makes the problem relatively simple. Our systems recommend content, and authors receive income when the content is used. They can answer users’ questions by <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/copyright-ai-and-provenance/" target="_blank">extracting text from content to which we’ve acquired the rights</a>; when we use AI to generate an answer, we know where that text has come from and can compensate the original author accordingly. These answers also link to the original source, where users can find more information, again generating income for the author. We don’t treat our authors and teachers as an undifferentiated class whose work we can repurpose at scale and without compensation. They aren’t abstractions who can be dissociated from the products of their labor.</p> <p>We encourage our authors and teachers to use AI responsibly and to work with us as we build new kinds of products to serve future generations of learners. We believe that using AI to create new products, while always keeping our responsibilities in mind, will generate more income for our talent pool—and that sticking to “business as usual,” the products that have worked in the past, isn’t to anyone’s advantage. Innovation in any technology, including training, entails risk. The alternative to risk-taking is stagnation. But the risks we take always account for our responsibilities to our partners: to compensate them fairly for their work and to build a learning platform on which they can prosper. In a future article, we will discuss our AI policies for our authors and our employees in more detail.</p> <p>The applications we are building are fairly clear-cut, and that clarity makes it relatively easy to establish rules for allocating income to authors. It’s less clear what a company’s responsibilities are when an AI isn’t simply extracting text but predicting the most likely next token one at a time. It’s important not to sidestep those issues either. It’s certainly conceivable that an AI could generate an introduction to a new programming language, borrowing some of the text from older content and generating new examples and discussions as necessary. Many programmers have already found <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://simonwillison.net/2022/Dec/5/rust-chatgpt-copilot/" target="_blank">ChatGPT a useful tool when learning a new language</a>. Tutorials could even be generated dynamically, at a user’s request. When an AI model is generating text by predicting the next token in the sequence, one token at a time, how do you attribute authorship?</p> <p>While it’s not yet clear how this will work out in practice, the principle is the same: generative AI doesn’t create new content; it extracts value from existing content, and the creators of that original content deserve compensation. It’s possible that these situations could be managed by careful prompting: for example, a system prompt or a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://research.ibm.com/blog/retrieval-augmented-generation-RAG" target="_blank">RAG</a> application that controls what sources are used to generate the answer would make attribution easier. Ignoring the issue and letting an AI generate text with no accountability isn’t a responsible solution. In this case, acting responsibly is about what you build as much as it is about who you pay; an ethical company builds systems that allow it to act responsibly. The current generation of models are, essentially, experiments that got out of control. It isn’t surprising that they don’t have all the features they need. But any models and applications built in the future will lack that excuse. </p> <p>Many other kinds of business partners will be affected by the use of AI: suppliers, wholesalers, retailers, contractors of many types. Some of these effects will result from their own use of AI; some won’t. But the principles of fairness and compensation where compensation is due remain the same. A company should not use AI to justify shortchanging its business partners.</p> <p>A company’s responsibilities to its business partners thus include:</p> <ul><li>Compensating business partners for all use of their content, including AI-repurposed content</li><li>Building applications that use AI to serve future generations of users</li><li>Encouraging partners to use AI responsibly in the products they develop</li></ul> <h2>Responsibilities to Customers</h2> <p>We all think we know what customers want: better products at lower prices, sometimes at prices that are below what’s reasonable. But that doesn’t take customers seriously. The first of O’Reilly’s <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/about/operating-principles.html" target="_blank">operating principles</a> is about customers—as are the next four. If a company wants to take its customers seriously, particularly in the context of AI-based products, what responsibilities should it be thinking about?</p> <p>Every customer must be treated with respect. Treating customers with respect starts with sales and customer service, two areas where AI is increasingly important. It’s important to build AI systems that aren’t abusive, even in subtle ways—even though human agents can also be abusive. But the responsibility extends much farther. Is a recommendation engine recommending appropriate products? We’ve certainly heard of Black women who only get recommendations for hair care products that white women use. We’ve also heard of Black men who see advertisements for bail bondsmen whenever they make any kind of a search. Is an AI system biased with respect to race, gender, or almost anything else? We don’t want real estate systems that reimplement redlining where minorities are only shown properties in certain areas. Will a résumé-screening system treat women and racial minorities fairly? Concern for bias goes even farther: it is possible for AI systems to develop bias against almost anything, including factors that it wouldn’t occur to humans to think about. Would we even know if an AI developed a bias against left-handed people?</p> <p>We’ve known for a long time that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/the-machine-learning-paradox/" target="_blank">machine learning systems can’t be perfect</a>. The tendency of the latest AI systems to hallucinate has only rubbed our faces in that fact. Although techniques like RAG can minimize errors, it is probably impossible to prevent them altogether, at least with the current generation of language models. What does that mean for our customers? They aren’t paying us for incorrect information at scale; at the same time, if they want AI-enhanced services, we can’t guarantee that all of AI’s results will be correct. Our responsibilities to customers for AI-driven products are threefold. We need to be honest that errors will occur; we need to use techniques that minimize the probability of errors; and we need to present (or be prepared to present) alternatives so they can use their judgment about which answers are appropriate to their situation.</p> <p>Respect for a customer includes respecting their privacy, an area in which online businesses are notably deficient. Any transaction involves a lot of data, ranging from data that’s essential to the transaction (what was bought, what was the price) to data that seems inconsequential but can still be collected and sold: browsing data obtained through cookies and tracking pixels is very valuable, and even arcana like keystroke timings can be collected and used to identify customers. Do you have the customer’s permission to sell the data that their transactions throw off? At least in the US, the laws on what you can do with data are porous and vary from state to state; because of GDPR, the situation in Europe is much clearer. But ethical and legal aren’t the same; “legal” is a minimum standard that many companies fail to meet. “Ethical” is about your own standards and principles for treating others responsibly and equitably. It is better to establish good principles that deal with your customers honestly and fairly than to wait for legislation to tell you what to do or to think that fines are just another expense of doing business. Does a company use data in ways that respect the customer? Would a customer be horrified to find out, after the fact, where their data has been sold? Would a customer be equally horrified to find that their conversations with AI have been <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.thestreet.com/technology/this-tech-giant-accidentally-just-leaked-sensitive-info-to-chatgpt" target="_blank">leaked</a> to other users?</p> <p>Every customer wants quality, but quality doesn’t mean the same thing to everyone. A customer on the edge of poverty might want durability rather than expensive fine fabrics—though the same customer might, on a different purchase, object to being pushed away from the more fashionable products they want. How does a company respect the customer’s wishes in a way that isn’t condescending and delivers a product that’s useful? Respecting the customer means focusing on what matters to them; and that’s true whether the agent working with the customer is a human or an AI. The kind of sensitivity required is difficult for humans and may be impossible for machines, but it’s no less essential. Achieving the right balance probably requires a careful collaboration between humans and AI.</p> <p>A business is also responsible for making decisions that are explainable. That issue doesn’t arise with human systems; if you are denied a loan, the bank can usually tell you why. (Whether the answer is honest may be another issue.) This isn’t true of AI, where explainability is still an active area for research. Some models are inherently explainable—for example, simple decision trees. There are explainability algorithms such as <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://homes.cs.washington.edu/~marcotcr/blog/lime/" target="_blank">LIME</a> that aren’t dependent on the underlying algorithm. Explainability for transformer-based AI (which includes just about all generative AI algorithms) is next to impossible. If explainability is a requirement—which is the case for almost anything involving money—it may be best to stay away from systems like ChatGPT. These systems make more sense in applications where explainability and correctness aren’t issues. Regardless of explainability, companies should <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2018/05/09/142959/a-new-company-audits-algorithms-to-see-how-biased-they-are/" target="_blank">audit</a> the outputs of AI systems to ensure that they’re fair and unbiased.</p> <p>The ability to explain a decision means little if it isn’t coupled with the ability to correct decisions. Respecting the customer means having a plan for redress. “The computer did it” was never a good excuse, and it’s even less acceptable now, especially since it’s widely known that AI systems of all types (not just natural language systems) generate errors. If an AI system improperly denies a loan, is it possible for a human to approve the loan anyway? Humans and AI need to learn how to work together—and AI should never be an excuse.</p> <p>Given this context, what are a company’s responsibilities to its customers? These responsibilities can be summed up with one word: respect. But respect is a very broad term; it includes:</p> <ul><li>Treating customers the way they would want to be treated</li><li>Respecting customers’ privacy</li><li>Understanding what the customer wants</li><li>Explaining decisions as needed</li><li>Providing a means to correct errors</li><li>Respecting customer privacy</li></ul> <h2>Responsibilities to Shareholders</h2> <p>It’s long been a cliché that a company’s primary responsibility is to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Shareholder_value" target="_blank">maximize shareholder value</a>. That’s a good pretext for arguing that a company has the right—no, the duty—to abuse employees, customers, and other stakeholders—particularly if the shareholder’s “value” is limited to the short-term. The idea that shareholder value is enshrined in law (either legislation or case law) is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://scholarship.law.cornell.edu/cgi/viewcontent.cgi?article=2311&context=facpub" target="_blank">apocryphal</a>. It appeared in the 1960s and 1970s and was propagated by Milton Friedman and the Chicago school of economics.</p> <p>Companies certainly have obligations to their shareholders, one of which is that shareholders deserve a return on their investment. But we need to ask whether this means short-term or long-term return. Finance in the US has fixated on short-term return, but that obsession is harmful to all of the stakeholders—except for executives who are often compensated in stock. When short-term returns cause a company to compromise the quality of its products, customers suffer. When short-term returns cause a company to layoff staff, the staff suffers, including those who stay: they are likely to be overworked and to fear further layoffs. Employees who fear losing their jobs, or are currently looking for new jobs, are likely to do a poor job of serving customers. Layoffs for strictly short-term financial gain are a vicious cycle for the company too: they lead to missed schedules, missed goals, and further layoffs. All of these result in a loss of credibility and poor long-term value. Indeed, one possible <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://qz.com/1776080/how-the-mcdonnell-douglas-boeing-merger-led-to-the-737-max-crisis" target="_blank">reason</a> for Boeing’s problems with the 737 Max and the 787 has been a shift from an engineering-dominated culture that focused on building the best product to a financial culture that focused on maximizing short-term profitability. If that theory is correct, the results of the cultural change are all too obvious and present a significant threat to the company’s future.</p> <p>What would a company that is truly responsible to its stakeholders look like, and how can AI be used to achieve that goal? We don’t have the right metrics; stock price, either short- or long-term, isn’t right. But we can think about what a corporation’s goals really are. O’Reilly’s operating principles start with the question “Is it best for the customer?” and continue with “Start with the customer’s point of view. It’s about them, not us.” Customer focus is a part of a company’s culture, and it’s antithetical to short-term returns. That doesn’t mean that customer focus sacrifices returns but that maximizing stock price leads to ways of thinking that aren’t in the customers’ interests. Closing a deal whether or not the product is right takes priority over doing right by the customer. We’ve all seen that happen; at one time or another, we’ve all been victims of it.</p> <p>There are many opportunities for AI to play a role in serving customers’ interests—and, in turn, serving shareholders’ interests. First, what does a customer want? Henry Ford probably didn’t say that customers want faster horses, but that remains an interesting observation. It’s certainly true that customers often don’t know what they really want or, if they do, can’t articulate it. Steve Jobs may have said that “our job is to figure out what they want before they do”; according to some stories, he <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.cultofmac.com/83222/steve-jobs-hides-in-bushes-to-spy-on-apple-customers/" target="_blank">lurked in the bushes</a> outside Apple’s Palo Alto store to watch customers’ reactions. Jobs’ secret weapon was intuition and imagination about what might be possible. Could AI help humans to discover what traditional custom research, such as focus groups (which Jobs hated), is bound to miss? Could an AI system with access to customer data (possibly including videos of customers trying out prototypes) <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.linkedin.com/pulse/steve-jobs-didnt-trust-conventional-customer-research-ryan-glasgow/" target="_blank">help humans develop the same kind of intuition</a> that Steve Jobs had? That kind of engagement between humans and AI goes beyond AI’s current capabilities, but it’s what we’re looking for. If a key to serving the customers’ interests is listening—really listening, not just recording—can AI be an aid without also become creepy and intrusive? Products that actually serve customers’ needs create long term value for all of the stakeholders.</p> <p>This is only one way in which AI can serve to drive long-term success and to help a business deliver on its responsibilities to stockholders and other stakeholders. The key, again, is collaboration between humans and AI, not using AI as a pretext for minimizing headcount or shortchanging product quality.</p> <p>It should go without saying, but in today’s business climate it doesn’t: one of a company’s responsibilities is to remain in business. Self-preservation at all costs is abusive, but a company that doesn’t survive isn’t doing its investors’ portfolios any favors. The US Chamber of Commerce, giving <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.uschamber.com/co/start/strategy/the-secret-to-long-term-business-success" target="_blank">advice</a> to small businesses, asks, “Have you created a dynamic environment that can quickly and effectively respond to market changes? If the answer is ‘no’ or ‘kind of,’ it’s time to get to work.” Right now, that advice means engaging with AI and deciding how to use it effectively and ethically. AI changes the market itself; but more than that, it is a tool for spotting changes early and thinking about strategies to respond to change. Again, it’s an area where success will require collaboration between humans and machines.</p> <p>Given this context, a company’s responsibility to its shareholders include:</p> <ul><li>Focusing on long-term rather than short-term returns</li><li>Building an organization that can respond to changes</li><li>Developing products that serve customers’ real needs</li><li>Enabling effective collaboration between humans and AI systems</li></ul> <h2>It’s About Honesty and Respect</h2> <p>A company has many stakeholders—not just the stockholders and certainly not just the executives. These stakeholders form a complex ecosystem. Corporate ethics is about treating all of these stakeholders, including employees and customers, responsibly, honestly, and with respect. It’s about balancing the needs of each group so that all can prosper, about taking a long-term view that realizes that a company can’t survive if it is only focused on short-term returns for stockholders. That has been a trap for many of the 20th century’s greatest companies, and it’s unfortunate that we see many technology companies traveling the same path. A company that builds products that aren’t fit for the market isn’t going to survive; a company that doesn’t respect its workforce will have trouble retaining good talent; and a company that doesn’t respect its business partners (in our case, authors, trainers, and partner publishers on our platform) will soon find itself without partners.</p> <p>Our corporate values demand that we do something better, that we keep the needs of all these constituencies in mind and in balance as we move our business forward. These values have nothing to do with AI, but that’s not surprising. AI creates ethical challenges, especially around the scale at which it can cause trouble when it is used inappropriately. However, it would be surprising if AI actually changed what we mean by honesty or respect. It would be surprising if the idea of behaving responsibly changed suddenly because AI became part of the equation.</p> <p>Acting responsibly toward your employees, customers, business partners, and stockholders: that’s the core of corporate ethics, with or without AI.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/corporate-responsibility-in-the-age-of-ai/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>The OpenAI Endgame</title> <link>https://www.oreilly.com/radar/the-openai-endgame/</link> <comments>https://www.oreilly.com/radar/the-openai-endgame/#respond</comments> <pubDate>Tue, 13 Feb 2024 11:07:40 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Commentary]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15401</guid> <description><![CDATA[Since the New York Times sued OpenAI for infringing its copyrights by using Times content for training, everyone involved with AI has been wondering about the consequences. How will this lawsuit play out? And, more importantly, how will the outcome affect the way we train and use large language models? There are two components to […]]]></description> <content:encoded><![CDATA[<p>Since the<em> New York Times</em> sued OpenAI for infringing its copyrights by using <em>Times</em> content for training, everyone involved with AI has been wondering about the consequences. How will this lawsuit play out? And, more importantly, how will the outcome affect the way we train and use large language models?</p> <p>There are two components to this suit. First, it was possible to get ChatGPT to reproduce some <em>Times</em> articles very close to verbatim. That’s fairly clearly copyright infringement, though there are still important questions that could influence the outcome of the case. Reproducing the<em> New York Times</em> clearly isn’t the intent of ChatGPT, and OpenAI appears to have modified ChatGPT’s guardrails to make generating infringing content more difficult, though probably not impossible. Is this enough to limit any damages? It’s not clear that anybody has used ChatGPT to avoid paying for a <em>NYT</em> subscription. Second, the examples in a case like this are always cherry-picked. While the <em>Times</em> can clearly show that OpenAI can reproduce some articles, can it reproduce any article from the <em>Times</em>’ archive? Could I get ChatGPT to produce an article from page 37 of the September 18, 1947 issue? Or, for that matter, an article from the<em> Chicago Tribune</em> or the<em> Boston Globe</em>? Is the entire corpus available (I doubt it), or just certain random articles? I don’t know, and given that OpenAI has modified GPT to reduce the possibility of infringement, it’s almost certainly too late to do that experiment. The courts will have to decide whether inadvertent, inconsequential, or unpredictable reproduction meets the legal definition of copyright infringement.</p> <p>The more important claim is that training a model on copyrighted content is infringement, whether or not the model is capable of reproducing that training data in its output. An inept and clumsy version of this claim was made by Sarah Silverman and others in a suit that was <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.hollywoodreporter.com/business/business-news/sarah-silverman-lawsuit-ai-meta-1235669403/" target="_blank">dismissed</a>. The Authors’ Guild has its own version of this lawsuit, and it is working on a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.hollywoodreporter.com/business/business-news/authors-guild-exploring-blanket-license-artificial-intelligence-companies-1235785941/" target="_blank">licensing</a> model that would allow its members to opt in to a single licensing agreement. The outcome of this case could have many side-effects, since it essentially would allow publishers to charge not just for the texts they produce, but for how those texts are used.</p> <p>It is difficult to predict what the outcome will be, though easy enough guess. Here’s mine. OpenAI will settle with the<em> New York Times</em> out of court, and we won’t get a ruling. This settlement will have important consequences: it will set a de-facto price on training data. And that price will no doubt be high. Perhaps not as high as the <em>Times</em> would like (there are rumors that OpenAI has offered something in the range of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.adweek.com/media/open-ai-response-new-york-times-lawsuit/#" target="_blank">$1 million to $5 million</a>), but sufficiently high enough to deter OpenAI’s competitors.</p> <p>$1M is not, in and of itself, a terribly high price, and the <em>Times</em> reportedly thinks that it’s way too low; but realize that OpenAI will have to pay a similar amount to almost every major newspaper publisher worldwide in addition to organizations like the Authors Guild, technical journal publishers, magazine publishers, and many other content owners. The total bill is likely to be close to $1 billion, if not more, and as models need to be updated, at least some of it will be a recurring cost. I suspect that OpenAI would have difficulty going higher, even given Microsoft’s investments—and, whatever else you may think of this strategy—OpenAI has to think about the total cost. I doubt that they are close to profitable; they appear to be running on an Uber-like business plan, in which they spend heavily to buy the market without regard for running a sustainable business. But even with that business model, billion-dollar expenses have to raise the eyebrows of partners like Microsoft.</p> <p>The <em>Times</em>, on the other hand, appears to be making a common mistake: overvaluing its data. Yes, it has a large archive—but what is the value of old news? Furthermore, in almost any application but especially in AI, the value of data isn’t the data itself; it’s the correlations between different datasets. The <em>Times</em> doesn’t own those correlations any more than I own the correlations between my browsing data and Tim O’Reilly’s. But those correlations are precisely what’s valuable to OpenAI and others building data-driven products.</p> <p>Having set the price of copyrighted training data to $1B or thereabouts, other model developers will need to pay similar amounts to license their training data: Google, Microsoft (for whatever independently developed models they have), Facebook, Amazon, and Apple. Those companies can afford it. Smaller startups (including companies like Anthropic and Cohere) will be priced out, along with every open source effort. By settling, OpenAI will eliminate much of their competition. And the good news for OpenAI is that even if they don’t settle, they still might lose the case. They’d probably end up paying more, but the effect on their competition would be the same. Not only that, the <em>Times</em> and other publishers would be responsible for enforcing this “agreement.” They’d be responsible for negotiating with other groups that want to use their content and suing those they can’t agree with. OpenAI keeps its hands clean, and its legal budget unspent. They can win by losing—and if so, do they have any real incentive to win?</p> <p>Unfortunately, OpenAI is right in claiming that a good model can’t be trained <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://the-decoder.com/openai-says-its-impossible-to-train-state-of-the-art-models-without-copyrighted-data/" target="_blank">without copyrighted data</a> (although Sam Altman, OpenAI’s CEO, has also said the <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.cnbc.com/2024/01/18/openai-ceo-on-nyt-lawsuit-ai-models-dont-need-publishers-data-.html" target="_blank">opposite</a>). Yes, we have substantial libraries of public domain literature, plus Wikipedia, plus papers in ArXiv, but if a language model trained on that data would produce text that sounds like a cross between 19th century novels and scientific papers, that’s not a pleasant thought. The problem isn’t just text generation; will a language model whose training data has been limited to copyright-free sources require prompts to be written in an early-20th or 19th century style? Newspapers and other copyrighted material are an excellent source of well-edited grammatically correct modern language. It is unreasonable to believe that a good model for modern languages can be built from sources that have fallen out of copyright.</p> <p>Requiring model-building organizations to purchase the rights to their training data would inevitably leave generative AI in the hands of a small number of unassailable monopolies. (We won’t address what can or can’t be done with copyrighted material, but we will say that copyright law says nothing at all about the source of the material: you can buy it legally, borrow it from a friend, steal it, find it in the trash—none of this has any bearing on copyright infringement.) One of the participants at the WEF roundtable <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.weforum.org/events/world-economic-forum-annual-meeting-2024/sessions/the-expanding-universe-of-generative-models/" target="_blank">The Expanding Universe of Generative Models</a> reported that Altman has said that he doesn’t see the need for more than one foundation model. That’s not unexpected, given my guess that his strategy is built around minimizing competition. But this is chilling: if all AI applications go through one of a small group of monopolists, can we trust those monopolists to deal honestly with issues of bias? AI developers have said a lot about “alignment,” but discussions of alignment always seem to sidestep more immediate issues like race and gender-based bias. Will it be possible to develop specialized applications (for example, O’Reilly Answers) that require training on a specific dataset? I’m sure the monopolists would say “of course, those can be built by fine tuning our foundation models”; but do we know whether that’s the best way to build those applications? Or whether smaller companies will be able to afford to build those applications, once the monopolists have succeeded in buying the market? Remember: Uber was once inexpensive.</p> <p>If model development is limited to a few wealthy companies, its future will be bleak. The outcome of copyright lawsuits won’t just apply to the current generation of Transformer-based models; they will apply to any model that needs training data. Limiting model building to a small number of companies will eliminate most academic research. It would certainly be possible for most research universities to build a training corpus on content they acquired legitimately. Any good library will have the <em>Times</em> and other newspapers on microfilm, which can be converted to text with OCR. But if the law specifies how copyrighted material can be used, research applications based on material a university has legitimately purchased may not be possible. It won’t be possible to develop open source models like <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/mistralai/Mistral-7B-v0.1" target="_blank">Mistral</a> and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/docs/transformers/model_doc/mixtral" target="_blank">Mixtral</a>—the funding to acquire training data won’t be there—which means that the smaller models that don’t require a massive server farm with power-hungry GPUs won’t exist. Many of these smaller models can run on a modern laptop, which makes them ideal platforms for developing AI-powered applications. Will that be possible in the future? Or will innovation only be possible through the entrenched monopolies?</p> <p>Open source AI has been the victim of a lot of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://spectrum.ieee.org/open-source-ai-2666932122" target="_blank">fear-mongering</a> lately. However, the idea that open source AI will be used irresponsibly to develop hostile applications that are inimical to human well-being gets the problem precisely wrong. Yes, open source will be used irresponsibly—as has every tool that has ever been invented. However, we know that hostile applications will be developed, and are already being developed: in military laboratories, in government laboratories, and at any number of companies. Open source gives us a chance to see what is going on behind those locked doors: to understand AI’s capabilities and possibly even to anticipate abuse of AI and prepare defenses. Handicapping open source AI doesn’t “protect” us from anything; it prevents us from becoming aware of threats and developing countermeasures.</p> <p>Transparency is important, and proprietary models will always lag open source models in transparency. Open source has always been about source code, rather than data; but that is changing. OpenAI’s GPT-4 scores surprisingly well on Stanford’s <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://crfm.stanford.edu/fmti/" target="_blank">Foundation Model Transparency Index</a>, but still lags behind the leading open source models (Meta’s LLaMA and BigScience’s BLOOM). However, it isn’t the total score that’s important; it’s the “upstream” score, which includes sources of training data, and on this the proprietary models aren’t close. Without data transparency, how will it be possible to understand biases that are built in to any model? Understanding those biases will be important to addressing the harms that models are doing now, not hypothetical harms that might arise from sci-fi superintelligence. Limiting AI development to a few wealthy players who make private agreements with publishers ensures that training data will never be open.</p> <p>What will AI be in the future? Will there be a proliferation of models? Will AI users, both corporate and individuals, be able to build tools that serve them? Or will we be stuck with a small number of AI models running in the cloud and being billed by the transaction, where we never really understand what the model is doing or what its capabilities are? That’s what the endgame to the legal battle between OpenAI and the <em>Times</em> is all about.</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/the-openai-endgame/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Radar Trends to Watch: February 2024</title> <link>https://www.oreilly.com/radar/radar-trends-to-watch-february-2024/</link> <comments>https://www.oreilly.com/radar/radar-trends-to-watch-february-2024/#respond</comments> <pubDate>Tue, 06 Feb 2024 11:01:51 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[Radar Trends]]></category> <category><![CDATA[Signals]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15392</guid> <description><![CDATA[2024 started with yet more AI: a small language model from Microsoft, a new (but unnamed) model from Meta that competes with GPT-4, and a text-to-video model from Google that claims to be more realistic than anything yet. Research into security issues has also progressed—unfortunately, discovering more problems than solutions. A common thread in several […]]]></description> <content:encoded><![CDATA[<p>2024 started with yet more AI: a small language model from Microsoft, a new (but unnamed) model from Meta that competes with GPT-4, and a text-to-video model from Google that claims to be more realistic than anything yet. Research into security issues has also progressed—unfortunately, discovering more problems than solutions. A common thread in several recent attacks has been to use embeddings: an attacker discovers innocuous text or images that happen to have an embedding similar to words describing actions that aren’t allowed. These innocuous inputs easily get by filters designed to prevent hostile prompts.</p> <h2>AI</h2> <ul><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://slgero.medium.com/merge-large-language-models-29897aeb1d1a" target="_blank">Merging large language models</a> gets developers the best of many worlds: use different models to solve different kinds of problems. It’s essentially <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Mixture_of_experts" target="_blank">mixture of experts</a> but applied at the application level of the stack rather than the model level.</li><li>Researchers have developed a method for <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2401.12070" target="_blank">detecting AI-generated text</a> that is 90% accurate and has a false positive rate of only 0.01%.</li><li>Google has announced <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://lumiere-video.github.io/" target="_blank">Lumiere</a>, a text-to-video model that generates “realistic, diverse, and coherent” motion. Lumiere generates the entire video in one pass rather than generating distinct keyframes that are then merged.</li><li>Is JavaScript a useful language for developing artificial intelligence applications? <em>The New Stack</em> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/top-5-javascript-tools-for-ai-engineering/" target="_blank">lists</a> five tools for building AI applications in JavaScript, starting with TensorFlow.js.</li><li>Meta has released a new language model that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://digialps.com/meta-research-introduces-revolutionary-self-rewarding-language-models-capable-of-gpt-4-level-performance/" target="_blank">claims performance similar to GPT-4</a>. It is a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/pdf/2401.10020.pdf" target="_blank">self-rewarding language model</a>; it continually evaluates its responses to prompts and adjusts its parameters in response. An independent open source <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/lucidrains/self-rewarding-lm-pytorch" target="_blank">implementation</a> is already on GitHub.</li><li>Hospitals are <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/health/2024/01/what-do-threads-mastodon-and-hospital-records-have-in-common/" target="_blank">using federated learning</a> techniques to collect and share patient data without compromising privacy. With federated learning, the hospitals aren’t sharing actual patient data but machine learning models built on local data.</li><li>Researchers have discovered “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2307.14539" target="_blank">compositional attacks</a>” against multimodal language models. In these attacks, prompts that combine text and images are used to “jailbreak” the model. A hostile but benign-looking image establishes a context in which the model ignores its guardrails.</li><li>Researchers have used tests for psychologically profiling humans to profile AI models and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2024-01-psychological-profiling-language-based-ai.html" target="_blank">research their built-in biases and prejudices</a>.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2305.18290" target="_blank">Direct Preference Optimization</a> (DPO) is an algorithm for training language models to operate in agreement with human preferences. It is simpler and more efficient than <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback" target="_blank">RLHF</a>.</li><li>Mistral has published a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2401.04088" target="_blank">paper</a> describing its Mixtral 8x7B model, a mixture of experts model with very impressive performance.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/cars/2024/01/volkswagen-is-adding-chatgpt-to-its-infotainment-system/" target="_blank">Volkswagen has added ChatGPT</a> to the infotainment system on its cars. ChatGPT will not have access to any of the car’s data.</li><li>Language models rely on converting input tokens to embeddings (long sequences of numbers). Can the original text be recovered from the embeddings used with language models? The answer may be <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2310.06816" target="_blank">yes</a>.</li><li>AWS’s AI product, Q, now has tools to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/aws-gifts-java-rust-developers-with-useful-tools/" target="_blank">automate updating Java programs</a> to new versions. That includes finding and replacing deprecated dependencies.</li><li>Microsoft’s Phi-2 model is now open source; it has been <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/microsoft/phi-2/commit/7e10f3ea09c0ebd373aebc73bc6e6ca58204628d" target="_blank">relicensed</a> with the MIT license. Phi-2 is a small model (2.7B parameters) with performance comparable to much larger models.</li><li>Simon Willison’s summary of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://simonwillison.net/2023/Dec/31/ai-in-2023/" target="_blank">AI in 2023</a> is the best we’ve seen. In the coming year, Simon would love to see us get beyond “vibes-based development.” Unlike traditional programming, AI doesn’t do what you tell it to do, and we’re frequently forced to evaluate AI output on the basis of whether it “feels right.”</li><li>The US FTC has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/ftc-offers-25-000-prize-for-detecting-ai-enabled-voice-cloning/" target="_blank">issued</a> a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.ftc.gov/news-events/contests/ftc-voice-cloning-challenge" target="_blank">challenge</a> to developers: develop software that can detect AI-generated clones of human voices. The winner will receive a $25,000 prize.</li><li>DeepMind has built a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/ai/2024/01/deepmind-ai-rivals-the-worlds-smartest-high-schoolers-at-geometry/" target="_blank">model that can solve geometry problems</a>. The new model combines a language model with symbolic AI, giving it the ability to reason logically about problems in addition to matching patterns.</li></ul> <h2>Programming</h2> <ul><li>Any app can become extensible. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://extism.org/" target="_blank">Extism</a> is a WebAssembly library that can be added to almost any app that allows app users to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/extism-v1-run-webassembly-in-your-app/" target="_blank">write plug-ins</a> in most major programming languages.</li><li>Zed, a collaborative code editor, is now <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://zed.dev/blog/zed-is-now-open-source" target="_blank">open source</a> and available on <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/zed-industries/zed" target="_blank">GitHub</a>.</li><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.blog/2024-01-23-good-devex-increases-productivity/" target="_blank">study</a> by GitHub shows that creating a good developer experience (DevEx or DX) improves productivity by reducing cognitive load, shortening feedback loops, and helping developers to remain in “flow state.”</li><li>Julia Evans (@<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="mailto:b0rk@jvns.ca" target="_blank">b0rk@jvns.ca</a>) has compiled a list of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://gist.github.com/jvns/f7d2db163298423751a9d1a823d7c7c1" target="_blank">common Git mistakes</a>.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://ruffle.rs/" target="_blank">Ruffle</a> is a Flash emulator built with Rust and Wasm. While you may not remember Macromedia Flash, and you probably don’t want to use it for new content, the New York Times is using Ruffle to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://flowingdata.com/2024/01/10/nyt-flash-based-visualizations-work-again/" target="_blank">resurrect</a> archival content that used Flash for visualizations.</li><li>JavaScript as a shell language? <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://bun.sh/blog/the-bun-shell" target="_blank">Bun</a> is an open source JavaScript shell that can run on Linux, macOS, and Windows. It’s the only shell that is truly platform-independent.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://shadeup.dev/" target="_blank">Shadeup</a> is a new programming language that extends TypeScript. It is designed to simplify working with WebGPU.</li><li>“<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/rethinking-observability/" target="_blank">Rethinking Observability</a>” argues for thinking about how users experience a service, rather than details of the service’s implementation. What are the critical user journeys (CUJs), and what are service level objectives (SLOs) for those paths through the system?</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://marimo.io/" target="_blank">Marimo</a> is a new Python notebook with some important features. When you edit any cell, it automatically updates all affected cells; the notebooks themselves are pure Python and can be managed with Git and other tools; GitHub Copilot is integrated into the Marimo editor.</li><li>LinkedIn has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/linkedin-shares-its-developer-productivity-framework/" target="_blank">released</a> its <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://linkedin.github.io/dph-framework/" target="_blank">Developer Productivity and Happiness Framework</a>, a set of metrics for processes that affect developer experience. The metrics include things like code review response time, but LinkedIn points out that the framework is most useful in helping teams build their own metrics.</li><li>The Node package registry, NPM, recently <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/everything-blocks-devs-from-removing-their-own-npm-packages/" target="_blank">accepted</a> a package named “everything” that links to everything in the registry. Whether this was a joke or a hostile attack remains to be seen, but an important side effect is that it became impossible to remove a package from NPM.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/ktock/container2wasm" target="_blank">container2wasm</a> takes a container image and converts it to WebAssembly, The Wasm executable can be run with <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://wasi.dev/" target="_blank">WASI</a> or even in a browser. This project is still in its early stages, but it is very impressive.</li><li>The <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://ahastack.dev/aha/1-stack-overview/" target="_blank">AHA Stack</a> provides a way to build web applications that minimizes browser-side JavaScript. It is based on the Astro framework, htmx, and Alpine.js.</li><li>Last year ended with Brainfuck implemented in PostScript. To start 2024, someone has found a working <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/kspalaiologos/malbolge-lisp" target="_blank">Lisp interpreter written in Malbolge</a>, a language that competes with Brainfuck for being the most difficult, frustrating, and obtuse programming language in existence.</li><li>The year starts with a new Python web framework, <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blog.miguelgrinberg.com/post/microdot-yet-another-python-web-framework" target="_blank">Microdot</a>. How long has it been since we’ve had a new Python framework? It’s very similar to Flask, but it’s small; it was designed to run on MicroPython, which runs on microcontrollers like ESP8266.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://odin-lang.org/" target="_blank">Odin</a> is yet another new programming language. It supports data-oriented programming and promises high performance with explicit (though safe) control of memory management and layout. It claims simplicity, clarity, and readability.</li></ul> <h2>Security</h2> <ul><li>The UK’s National Cyber Security Center has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/uk-says-ai-will-empower-ransomware-over-the-next-two-years/" target="_blank">warned</a> that generative AI will be used in ransomware and other attacks. Generative AI will make social engineering and phishing more convincing; it will enable inexperienced actors to create much more dangerous attacks.</li><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.schneier.com/blog/archives/2024/01/side-channels-are-common.html" target="_blank">presentation at USENIX’s security symposium</a> argues that side channels leak information in almost all commodity PCs: microphones, cameras, and other sensors pick up electromagnetic signals from the processor. These signals can be captured and decoded.</li><li>Like everyone else, malware groups are <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theregister.com/2023/12/11/lazarus_group_edang/" target="_blank">moving to memory-safe languages</a> like Rust and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://dlang.org/" target="_blank">DLang</a> to develop their payloads.</li><li>Researchers have <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2401.05566" target="_blank">discovered</a> that poisoned training data can be used to insert backdoors into large language models. These backdoors can be triggered by special prompts and cannot be discovered or removed by current safety techniques.</li><li>Programmers who use AI assistants are likely to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2211.03622" target="_blank">write code that is less secure</a> while believing that their code is more secure. However, users of AI assistants who don’t “trust” the AI engage more with the code produced and are likely to produce code that is more secure.</li><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2024/01/a-previously-unknown-worm-has-been-stealthily-targeting-linux-devices-for-a-year/" target="_blank">variant of the Mirai malware is attacking Linux systems</a>. This variant finds weak SSH passwords and installs cryptocurrency mining software to create a mining botnet.</li><li>Many groups offer “bug bounties” that pay rewards to those who discover bugs (particularly security vulnerabilities) in their code. One open source maintainer <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-for-intelligence/" target="_blank">argues</a> that this process is being distorted by incorrect bug reports that are generated by AI, wasting maintainers’ time.</li><li>The US National Institute of Standards and Technology <a href="https://venturebeat.com/security/new-nist-report-sounds-the-alarm-on-growing-threat-of-ai-attacks/">has</a> <a href="https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-2e2023.pdf">published</a> a taxonomy and standard terminology for attacks against machine learning and AI systems.</li></ul> <h2>Web</h2> <ul><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://nimbo.earth/products/earth-online/" target="_blank">Nimbo Earth Online</a> aims to be a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenextweb.com/news/digital-twin-rival-google-earth-nimbo" target="_blank">“digital twin” of the Earth</a>. It’s superficially similar to Google Earth but has fascinating features like the ability to see historical progressions: for example, how a landscape changed after a fire or how a river’s course wandered over the years.</li><li>A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://downloads.webis.de/publications/papers/bevendorff_2024a.pdf" target="_blank">study</a> shows that search results are getting worse as a result of SEO spam. The problem affects all major search engines. If you read the paper and ignore click-bait summaries, Google is doing a somewhat better job of maintaining search integrity than its competitors.</li><li><em>The Verge</em> has an excellent <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theverge.com/c/23998379/google-search-seo-algorithm-webpage-optimization" target="_blank">article</a> about how optimizing sites for Google search have affected web design, making sites much more homogeneous.</li><li>Facebook’s app includes a new <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://gizmodo.com/meet-link-history-facebook-s-new-way-to-track-the-we-1851134018" target="_blank">Link History</a> setting (on by default) that encourages use of the app’s built-in browser. Link History saves all links, and the browser is known to include a keylogger; the data from both is used for targeted advertising.</li></ul> <h2>Quantum Computing</h2> <ul><li>While we don’t yet have usable quantum computers, an <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.schneier.com/blog/archives/2024/01/improving-shors-algorithm.html" target="_blank">improvement</a> to Shor’s algorithm for factoring numbers has been <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2308.06572" target="_blank">published</a>. While it reduces the computational time from O(N^2) to O(N^1.5), it increases the number of qubits required, which may be an important limitation.</li></ul>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/radar-trends-to-watch-february-2024/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Technology Trends for 2024</title> <link>https://www.oreilly.com/radar/technology-trends-for-2024/</link> <comments>https://www.oreilly.com/radar/technology-trends-for-2024/#respond</comments> <pubDate>Thu, 25 Jan 2024 11:04:43 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[Radar Column]]></category> <category><![CDATA[Research]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15363</guid> <description><![CDATA[This has been a strange year. While we like to talk about how fast technology moves, internet time, and all that, in reality the last major new idea in software architecture was microservices, which dates to roughly 2015. Before that, cloud computing itself took off in roughly 2010 (AWS was founded in 2006); and Agile […]]]></description> <content:encoded><![CDATA[<p>This has been a strange year. While we like to talk about how fast technology moves, internet time, and all that, in reality the last major new idea in software architecture was microservices, which dates to roughly 2015. Before that, cloud computing itself took off in roughly 2010 (AWS was founded in 2006); and Agile goes back to 2000 (the <em>Agile Manifesto</em> dates back to 2001, Extreme Programming to 1999). The web is over 30 years old; the Netscape browser appeared in 1994, and it wasn’t the first. We think the industry has been in constant upheaval, but there have been relatively few disruptions: one every five years, if that.</p> <p>2023 was one of those rare disruptive years. ChatGPT changed the industry, if not the world. We’re skeptical about things like job displacement, at least in technology. But AI is going to bring changes to almost every aspect of the software industry. What will those changes be? We don’t know yet; we’re still at the beginning of the story. In this report about how people are using O’Reilly’s learning platform, we’ll see how patterns are beginning to shift.</p> <p>Just a few notes on methodology: This report is based on O’Reilly’s internal “Units Viewed” metric. Units Viewed measures the actual usage of content on our platform. The data used in this report covers January through November in 2022 and 2023. Each graph is scaled so that the topic with the greatest usage is 1. Therefore, the graphs can’t be compared directly to each other.</p> <p>Remember that these “units” are “viewed” by our users, who are largely professional software developers and programmers. They aren’t necessarily following the latest trends. They’re solving real-world problems for their employers. And they’re picking up the skills they need to advance in their current positions or to get new ones. We don’t want to discount those who use our platform to get up to speed on the latest hot technology: that’s how the industry moves forward. But to understand usage patterns, it’s important to realize that every company has its own technology stacks, and that those stacks change slowly. Companies aren’t going to throw out 20 years’ investment in PHP so they can adopt the latest popular React framework, which will probably be displaced by another popular framework next year.</p> <h2>Software Development</h2> <p>Most of the topics that fall under software development declined in 2023. What does this mean? Programmers are still writing software; our lives are increasingly mediated by software, and that isn’t going to change.</p> <p>Software developers are responsible for designing and building bigger and more complex projects than ever. That’s one trend that won’t change: complexity is always “up and to the right.” Generative AI is the wild card: Will it help developers to manage complexity? Or will it add complexity all its own? It’s tempting to look at AI as a quick fix. Who wants to learn about coding practices when you’re letting GitHub Copilot write your code for you? Who wants to learn about design patterns or software architecture when some AI application may eventually do your high-level design? AI is writing low-level code now; as many as <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.blog/2023-06-13-survey-reveals-ais-impact-on-the-developer-experience/#:~:text=on%20developing%20solutions.-,The%20bottom%20line,but%20enable%20upskilling%20opportunities%2C%20too." target="_blank">92% of software developers are using it</a>. Whether it will be able to do high-level design is an open question—but as always, that question has two sides: “Will AI do our design work?” is less interesting than “How will AI change the things we want to design?” And the real question that will change our industry is “How do we design systems in which generative AI and humans collaborate effectively?”</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig1-678x1048.png" alt="" class="wp-image-15364" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig1-678x1048.png 678w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig1-194x300.png 194w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig1-768x1187.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig1-994x1536.png 994w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig1.png 1209w" sizes="(max-width: 678px) 100vw, 678px" /><figcaption>Figure 1. Software architecture</figcaption></figure> <p>Regardless of the answers to these questions, humans will need to understand and specify what needs to be designed. Our data shows that most topics in software architecture and design are down year-over-year. But there are exceptions. While software architecture is down 3.9% (a relatively small decline), enterprise architecture is up 8.9%. Domain-driven design is particularly useful for understanding the behavior of complex enterprise systems; it’s down, but only 2.0%. Use of content about event-driven architecture is relatively small, but it’s up 40%. That change is important because event-driven architecture is a tool for designing large systems that have to ingest data from many different streams in real time. Functional programming, which many developers see as a design paradigm that will help solve the problems of distributed systems, is up 9.8%. So the software development world is changing. It’s shifting toward distributed systems that manage large flows of data in real time. Use of content on topics relevant to that shift is holding its own or growing.</p> <p>Microservices saw a 20% drop. Many developers expressed frustration with microservices during the year and argued for a return to monoliths. That accounts for the sharp decline—and it’s fair to say that many organizations are paying the price for moving to microservices because it was “the thing to do,” not because they needed the scale or flexibility that microservices can offer. From the start, microservice proponents have argued that the best way to develop microservices is to start with a monolith, then break the monolith into services as it becomes necessary. If implemented poorly, microservices deliver neither scale nor flexibility. Microservices aren’t ideal for new greenfield projects, unless you’re absolutely sure that you need them from the start—and even then, you should think twice. It’s definitely not a technology to implement just to follow the latest fad.</p> <p>Software developers run hot and cold on design patterns, which declined 16%. Why? It probably depends on the wind or the phase of the moon. Content usage about design patterns increased 13% from 2021 to 2022, so this year’s decline just undoes last year’s gain. It’s possible that understanding patterns seems less important when AI is writing a lot of the code for you. It’s also possible that design patterns seem less relevant when code is already largely written; most programmers maintain existing applications rather than develop new greenfield apps, and few texts about design patterns discuss the patterns that are embedded in legacy applications. But both ways of thinking miss the point. Design patterns are common solutions to common problems that have been observed in practice. Understanding design patterns keeps you from reinventing wheels. Frameworks like React and Spring are important because they implement design patterns. Legacy applications won’t be improved by refactoring existing code just to use some pattern, but design patterns are useful for extending existing software and making it more flexible. And, of course, design patterns are used in legacy code—even code that was written before the term was coined! Patterns are discovered, not “invented”; again, they’re common solutions to problems programmers have been solving since the beginning of programming.</p> <p>At the same time, whenever there’s a surge of interest in design patterns, there’s a corresponding surge in pattern abuse: managers asking developers how many patterns they used (as if pattern count were a metric for good code), developers implementing FactoryFactoryFactory Factories, and the like. What goes around comes around, and the abuse of design patterns is part of a feedback loop that regulates the use of design patterns.</p> <h2>Programming and Programming Languages</h2> <p>Most of the programming languages we track showed declines in content usage. Before discussing specifics, though, we need to look at general trends. If 92% of programmers are using generative AI to write code and answer questions, then we’d certainly expect a drop in content use. That may or may not be advisable for career development, but it’s a reality that businesses built on training and learning have to acknowledge. But that isn’t the whole story either—and the bigger story leaves us with more questions than answers.</p> <p>Rachel Stephens provides two fascinating pieces of the puzzle in a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://redmonk.com/rstephens/2023/12/14/language-rankings-update/" target="_blank">recent article on the RedMonk blog</a>, but those pieces don’t fit together exactly. First, she notes the decline in questions asked on Stack Overflow and states (reasonably) that asking a nonjudgmental AI assistant might be a preferable way for beginners to get their questions answered. We agree; we at O’Reilly have built <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/online-learning/feature-answers.html" target="_blank">O’Reilly Answers</a> to provide that kind of assistance (and are in the process of a major upgrade that will make it even more useful). But Stack Overflow shows a broad peak in questions from 2014 to 2017, with a sharp decline afterward; the number of questions in 2023 is barely 50% of the peak, and the 20% decline from the January 2023 report to the July report is only somewhat sharper than the previous drops. And there was no generative AI, no ChatGPT, back in 2017 when the decline began. Did generative AI play a role? It would be foolish to say that it didn’t, but it can’t be the whole story.</p> <p>Stephens points to another anomaly: GitHub pull requests declined roughly 25% from the second half of 2022 to the first half of 2023. Why? Stephens guesses that there was increased GitHub activity during the pandemic and that activity has returned to normal now that we’ve (incorrectly) decided the pandemic is over. Our own theory is that it’s a reaction to GPT models leaking proprietary code and abusing open source licenses; that could cause programmers to be wary of public code repositories. But those are only guesses. This change is apparently not an error in the data. It might be a one-time anomaly, but no one really knows the cause. <em>Something</em> drove down programmer activity on GitHub, and that’s inevitably a part of the background to this year’s data.</p> <p>So, what does O’Reilly’s data say? As it has been for many years, Python is the most widely used programming language on our platform. This year, we didn’t see an increase; we saw a very small (0.14%) decline. That’s noise; we won’t insult your intelligence by claiming that “flat in a down market” is really a gain. It’s certainly fair to ask whether a language as popular as Python has gathered all the market share that it will get. When you’re at the top of the adoption curve, it’s difficult to go any higher and much easier to drop back. There are always new languages ready to take some of Python’s market share. The most significant change in the Python ecosystem is Microsoft’s integration of Python into Excel spreadsheets, but it’s too early to expect that to have had an effect.</p> <p>Use of content about Java declined 14%, a significant drop but not out of line with the drop in GitHub activity. Like Python, Java is a mature language and may have nowhere to go but down. It has never been “well loved”; when Java was first announced, people walked out of the doors of the conference room claiming that Java was dead before you could even download the beta. (I was there.) Is it time to dance on Java’s grave? That dance has been going on since 1995, and it hasn’t been right yet.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig2-975x1048.png" alt="" class="wp-image-15365" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig2-975x1048.png 975w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig2-279x300.png 279w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig2-768x825.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig2.png 1209w" sizes="(max-width: 975px) 100vw, 975px" /><figcaption>Figure 2. Programming languages</figcaption></figure> <p>JavaScript also declined by 3.9%. It’s a small decline and probably not meaningful. TypeScript, a version of JavaScript that adds static typing and type annotations, gained 5.6%. It’s tempting to say that these cancel each other out, but that’s not correct. Usage of TypeScript content is roughly one-tenth the usage of JavaScript content. But it is correct to say that interest in type systems is growing among web developers. It’s also true that an increasing number of junior developers use JavaScript only through a framework like React or Vue. Boot camps and other crash programs often train students in “React,” with little attention on the bigger picture. Developers trained in programs like these may be aware of JavaScript but may not think of themselves as JavaScript developers, and may not be looking to learn more about the language outside of a narrow, framework-defined context.</p> <p>We see growth in C++ (10%), which is surprising for an old, well-established language. (C++ first appeared in 1985.) At this point in C++’s history, we’d expect it to be a headache for people maintaining legacy code, not a language for starting new projects. Why is it growing? While C++ has long been an important language for game development, there are signs that it’s breaking out into other areas. C++ is an ideal language for embedded systems, which often require software that runs directly on the processor (for example, the software that runs in a smart lightbulb or in the braking system of any modern car). You aren’t going to use Python, Java, or JavaScript for those applications. C++ is also an excellent language for number crunching (Python’s numeric libraries are written in C++), which is increasingly important as artificial intelligence goes mainstream. It has also become the new “must have” language on résumés: knowing C++ proves that you’re tough, that you’re a “serious” programmer. Job anxiety exists—whether or not it’s merited is a different question—and in an environment where programmers are nervous about keeping their current jobs or looking forward to finding a new one, knowing a difficult but widely used language can only be an asset.</p> <p>Use of content about Rust also increased from 2022 to 2023 (7.8%). Rust is a relatively young language that stresses memory safety and performance. While Rust is considered difficult to learn, the idea that memory safety is baked in makes it an important alternative to languages like C++. Bugs in memory management are a significant source of vulnerabilities, as noted in NIST’s page on “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.nist.gov/itl/ssd/software-quality-group/safer-languages" target="_blank">Safer Languages</a>,” and Rust does a good job of enforcing safe memory usage. It’s now used in operating systems (Linux kernel components), tool development, and even enterprise software.</p> <p>We also saw 9.8% growth in content about functional programming. We didn’t see gains for any of the historical functional programming languages (Haskell, Erlang, Lisp, and Elixir) though; most saw steep declines. In the past decade, most programming languages have added functional features. Newer languages like Rust and Go have had them from the start. And Java has gradually added features like closures in a series of updates. Now programmers can be as functional as they want to be without switching to a new language.</p> <p>Finally, there are some programming languages that we don’t yet track but that we’re watching with interest. Zig is a simple imperative language that’s designed to be memory safe, like Rust, but relatively easy to learn. Mojo is a superset of Python that’s compiled, not interpreted. It’s designed for high performance, especially for numerical operations. Mojo’s goal is to facilitate AI programming in a single language rather than a combination of Python and some other language (typically C++) that’s used for performance-critical numerical code. Where are these languages going? It will be some years before they reach the level of Rust or Go, but they’re off to a good start.</p> <p>So what does all this tell us about training and skill development? It’s easy to think that, with Copilot and other tools to answer all your questions, you don’t need to put as much effort into learning new technologies. We all ask questions on Google or Stack Overflow, and now we have other places to get answers. Necessary as that is, the idea that asking questions can replace training is naive. Unlike many who are observing the influence of generative AI on programming, we believe that it will increase the gap between entry-level skills and senior developer skills. Being a senior developer—being a senior anything—requires a kind of fluency that you can’t get just from asking questions. I may never be a fluent user of Python’s pandas library (which I used extensively to write this report); I asked lots of questions, and that has undoubtedly saved me time. But what happens when I need to solve the next problem? The kind of fluency that you need to look at a problem and understand how to solve it doesn’t come from asking simple “How do I do this?” questions. Nor does it preclude asking lots of “I forgot how this function works” questions. That’s why we’ve built <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/answers/search/" target="_blank">O’Reilly Answers</a>, an AI-driven service that finds solutions to questions using content from our platform. But expertise does require developing the intellectual muscle that comes from grappling with problems and solving them yourself rather than letting something else solve them for you. (And that includes forcing yourself to remember all the messy syntax details.) People who think generative AI is a shortcut to expertise (and the job title and salary that expertise merits) are shortchanging themselves.</p> <h2>Artificial Intelligence</h2> <p>In AI, there’s one story and only one story, and that’s the GPT family of models. Usage of content on these models exploded 3,600% in the past year. That explosion is tied to the appearance of ChatGPT in November 2022. But don’t make the mistake of thinking that ChatGPT came out of nowhere. GPT-3 created a big splash when it was released in 2020 (complete with a clumsy web-based interface). GPT-2 appeared in 2019, and the original unnumbered GPT was even earlier. The real innovation in ChatGPT wasn’t the technology itself (though the models behind it represent a significant breakthrough in AI performance); it was packaging the model as a chatbot. That doesn’t mean that the GPT explosion wasn’t real. While our analysis of search trends shows that interest in ChatGPT has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/the-chatgpt-surge/" target="_blank">peaked</a> among our platform’s users, interest in natural language processing (NLP) showed a 195% increase—and from a much higher starting point.<sup>1</sup> That makes sense, given the more technical nature of our audience. Software developers will be building on top of the APIs for GPT and other language models and are likely less interested in ChatGPT, the web-based chat service. Related topics generative models (900%) and Transformers (325%) also showed huge gains. Prompt engineering, which didn’t exist in 2022, became a significant topic, with roughly the same usage as Transformers. As far as total use, NLP is almost twice GPT. However you want to read the data, this is AI’s big year, largely due to the GPT models and the idea of generative AI.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig3-680x1048.png" alt="" class="wp-image-15366" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig3-680x1048.png 680w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig3-195x300.png 195w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig3-768x1183.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig3-997x1536.png 997w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig3.png 1209w" sizes="(max-width: 680px) 100vw, 680px" /><figcaption>Figure 3. Artificial intelligence</figcaption></figure> <p>But don’t assume that the explosion of interest in generative AI meant that other aspects of AI were standing still. Deep learning, the creation and application of neural networks with many layers, is fundamental to every aspect of modern AI. Usage in deep learning content grew 19% in the past year. Reinforcement learning, in which models are trained by giving “rewards” for solving problems, grew 15%. Those gains only look small in comparison to the triple- and quadruple-digit gains we’re seeing in natural language processing. PyTorch, the Python library that has come to dominate programming in machine learning and AI, grew 25%. In recent years, interest in PyTorch has been growing at the expense of TensorFlow, but TensorFlow showed a small gain (1.4%), reversing (or at least pausing) its decline. Interest in two older libraries, scikit-learn and Keras, declined: 25% for scikit-learn and 4.8% for Keras. Keras has largely been subsumed by TensorFlow, while scikit-learn hasn’t yet incorporated the capabilities that would make it a good platform for building generative AI. (An attempt to implement Transformers in scikit-learn appears to be underway at <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/scikit-learn/sklearn-transformers" target="_blank">Hugging Face</a>.)</p> <p>We’ve long said that operations is the elephant in the room for machine learning and artificial intelligence. Building models and developing applications is challenging and fun, but no technology can mature if IT teams can’t deploy, monitor, and manage it. Interest in operations for machine learning (MLOps) grew 14% over the past year. This is solid, substantial growth that only looks small in comparison with topics like generative AI. Again, we’re still in the early stages—generative AI and large language models are only starting to reach production. If anything, this increase probably reflects older applications of AI. There’s a growing ecosystem of startups building tools for deploying and monitoring language models, which are fundamentally different from traditional applications. As companies deploy the applications they’ve been building, MLOps will continue to see solid growth. (More on MLOps when we discuss operations below.)</p> <p><a href="https://www.langchain.com/">LangChain</a> is a framework for building generative AI applications around groups of models and databases. It’s often used to implement the <a href="https://thenewstack.io/retrieval-augmented-generation-for-llms/">retrieval-augmented generation (RAG) pattern</a>, where a user’s prompt is used to look up relevant items in a vector database; those items are then combined with the prompt, generating a new prompt that is sent to the language model. There isn’t much content about LangChain available yet, and it didn’t exist in 2022, but it’s clearly going to become a foundational technology. Likewise, vector databases aren’t yet in our data. We expect that to change next year. They are rather specialized, so we expect usage to be relatively small, unlike products like MySQL—but they will be very important.</p> <p>AI wasn’t dominated entirely by the work of OpenAI; Meta’s LLaMA and Llama 2 also attracted a lot of attention. The source code for LLaMA was open source, and its weights (parameters) were easily available to researchers. Those weights quickly leaked from “researchers” to the general public, where they jump-started the creation of smaller open source models. These models are much smaller than behemoths like GPT-4. Many of them can run on laptops, and they’re proving ideal for smaller companies that don’t want to rely on Microsoft, OpenAI, or Google to provide AI services. (If you want to run an open source language model on your laptop, try <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/Mozilla-Ocho/llamafile" target="_blank">llamafile</a>.) While huge “foundation models” like the GPT family won’t disappear, in the long run open source models like Alpaca and Mistral may prove to be more important to software developers.</p> <p>It’s easy to think that generative AI is just about software development. It isn’t; its influence extends to just about every field. Our <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/videos/chatgpt-possibilities-and/0636920908753/" target="_blank">ChatGPT: Possibilities and Pitfalls</a> Superstream was the most widely attended event we’ve ever run. There were over 28,000 registrations, with attendees and sponsors from industries as diverse as pharmaceuticals, logistics, and manufacturing. Attendees included small business owners, sales and marketing personnel, and C-suite executives, along with many programmers and engineers from different disciplines. We’ve also been running courses focused on specific industries: <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/videos/generative-ai-for/0636920962335/" target="_blank">Generative AI for Finance</a> had over 2,000 registrations, and <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/videos/generative-ai-for/0636920964384/" target="_blank">Generative AI for Government</a> over 1,000. And more than 1,000 people signed up for our <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/live-events/generative-ai-for-healthcare/0636920098725/" target="_blank">Generative AI for Healthcare</a> event.</p> <h2>Data</h2> <p>In previous years, we would have told the story of AI as part of the story of data. That’s still correct; with its heavy emphasis on mathematics and statistics, AI is a natural outgrowth of data science. But this year, AI has become the superstar that gets top billing, while data is a supporting actor.</p> <p>That doesn’t mean that data is unimportant. Far from it. Every company uses data: for planning, for making projections, for analyzing what’s happening within the business and the markets they serve. So it’s not surprising that the second biggest topic in data is Microsoft Power BI, with a 36% increase since 2022. SQL Server also showed a 5.3% increase, and statistics toolbox R increased by 4.8%.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig4-692x1048.png" alt="" class="wp-image-15367" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig4-692x1048.png 692w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig4-198x300.png 198w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig4-768x1162.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig4-1015x1536.png 1015w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig4.png 1209w" sizes="(max-width: 692px) 100vw, 692px" /><figcaption>Figure 4. Data analysis and databases</figcaption></figure> <p>Data engineering was by far the most heavily used topic in this category; it showed a 3.6% decline, stabilizing after a huge gain from 2021 to 2022. Data engineering deals with the problem of storing data at scale and delivering that data to applications. It includes moving data to the cloud, building pipelines for acquiring data and getting data to application software (often in near real time), resolving the issues that are caused by data siloed in different organizations, and more. Two of the most important platforms for data engineering, Kafka and Spark, showed significant declines in 2023 (21% and 20%, respectively). Kafka and Spark have been workhorses for many years, but they are starting to show their age as they become “legacy technology.” (Hadoop, down 26%, is clearly legacy software in 2023.) Interest in Kafka is likely to rise as AI teams start implementing real-time models that have up-to-the-minute knowledge of external data. But we also have to point out that there are newer streaming platforms (like Pulsar) and newer data platforms (like Ray).</p> <p>Designing enterprise-scale data storage systems is a core part of data engineering. Interest in data warehouses saw an 18% drop from 2022 to 2023. That’s not surprising; data warehouses also qualify as legacy technology. Two other patterns for enterprise-scale storage show significant increases: Usage of content about data lakes is up 37% and, in absolute terms, significantly higher than that of data warehouses. Usage for data mesh content is up 5.6%. Both lakes and meshes solve a basic problem: How do you store data so that it’s easy to access across an organization without building silos that are only relevant to specific groups? <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Data_lake" target="_blank">Data lakes</a> can include data in many different formats, and it’s up to users to supply structure when data is utilized. A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.montecarlodata.com/blog-what-is-a-data-mesh-and-how-not-to-mesh-it-up/" target="_blank">data mesh</a> is a truly distributed solution: each group is responsible for its own data but makes that data available throughout the enterprise through an interoperability layer. Those newer technologies are where we see growth.</p> <p>The two open source data analysis platforms were virtually unchanged in 2023. Usage of content about R increased by 3.6%; we’ve already seen that Python was unchanged, and pandas grew by 1.4%. Neither of these is going anywhere, but alternatives, particularly to pandas, are appearing.</p> <h2>Operations</h2> <p>Whether you call it operations, DevOps, or something else, this field has seen some important changes in the past year. We’ve witnessed the rise of developer platforms, along with the related topic, platform engineering. Both of those are too new to be reflected in our data: you can’t report content use before content exists. But they are influencing other topics.</p> <p>We’ve said in the past that Linux is table stakes for a job in IT. That’s still true. But the more the deployment process is automated—and platform engineering is just the next step in “Automate All the Things”—the less developers and IT staff need to know about Linux. Software is packaged in containers, and the containers themselves run as virtual Linux instances, but developers don’t need to know how to find and kill out-of-control processes, do a backup, install device drivers, or perform any of the other tasks that are the core of system administration. Usage of content about Linux is down 6.9%: not a major change but possibly a reflection of the fact that the latest steps forward in deploying and managing software shield people from direct contact with the operating system.</p> <p>Similar trends reduce what developers and IT staff need to know about Kubernetes, the near-ubiquitous container orchestrator (down 6.9%). Anyone who uses Kubernetes knows that it’s complex. We’ve long expected “something simpler” to come along and replace it. It hasn’t—but again, developer platforms put users a step further away from engaging with Kubernetes itself. Knowledge of the details is encapsulated either in a developer platform or, perhaps more often, in a Kubernetes service administered by a cloud provider. Kubernetes can’t be ignored, but it’s more important to understand high-level principles than low-level commands.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig5-702x1048.png" alt="" class="wp-image-15368" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig5-702x1048.png 702w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig5-201x300.png 201w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig5-768x1147.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig5-1028x1536.png 1028w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig5.png 1209w" sizes="(max-width: 702px) 100vw, 702px" /><figcaption>Figure 5. Infrastructure and operations</figcaption></figure> <p>DevOps (9.0%) and SRE (13%) are also down, though we don’t think that’s significant. Terms come and go, and these are going. While operations is constantly evolving, we don’t believe we’ll ever get to the mythical state of “NoOps,” nor should we. Instead, we’ll see constant evolution as the ratio of systems managed to operations staff grows ever higher. But we <em>do</em> believe that sooner rather than later, someone will put a new name on the disciplines of DevOps and its close relative, SRE. That new name might be “platform engineering,” though that term says more about designing deployment pipelines than about carrying the pager and keeping the systems running; platform engineering is about treating developers as customers and designing internal developer platforms that make it easy to test and deploy software systems with minimal ceremony. We don’t believe that platform engineering subsumes or replaces DevOps. Both are partners in improving experience for developers and operations staff (and ratcheting up the ratio of systems managed to staff even higher).</p> <p>That’s a lot of red ink. What’s in the black? Supply chain management is up 5.9%. That’s not a huge increase, but in the past few years we’ve been forced to think about how we manage the software supply chain. Any significant application easily has dozens of dependencies, and each of those dependencies has its own dependencies. The total number of dependencies, including both direct and inherited dependencies, can easily be hundreds or even thousands. Malicious operators have discovered that they can corrupt software archives, getting programmers to inadvertently incorporate malware into their software. Unfortunately, security problems never really go away; we expect software supply chain security to remain an important issue for the foreseeable (and unforeseeable) future.</p> <p>We’ve already mentioned that MLOps, the discipline of deploying and managing models for machine learning and artificial intelligence, is up 14%. Machine learning and AI represent a new kind of software that doesn’t follow traditional rules, so traditional approaches to operations don’t work. The list of differences is long:</p> <ul><li>While most approaches to deployment are based on the idea that an application can be reproduced from a source archive, that isn’t true for AI. An AI system depends as much on the training data as it does on the source code, and we don’t yet have good tools for archiving training data.</li><li>While we’ve said that open source models such as Alpaca are much smaller than models like GPT-4 or Google’s Gemini, even the smallest of those models is very large by any reasonable standard.</li><li>While we’ve gotten used to automated testing as part of a deployment pipeline, AI models aren’t deterministic. A test doesn’t necessarily give the same result every time it runs. Testing is no less important for AI than it is for traditional software (arguably it’s more important), and we’re starting to see startups built around AI testing, but we’re still at the beginning.</li></ul> <p>That’s just a start. MLOps is a badly needed specialty. It’s good to see growing interest.</p> <h2>Security</h2> <p>Almost all branches of security showed growth from 2022 to 2023. That’s a welcome change: in the recent past, many companies talked about security but never made the investment needed to secure their systems. That’s changing, for reasons that are obvious to anyone who reads the news. Nobody wants to be a victim of data theft or ransomware, particularly now that ransomware has evolved into blackmail.</p> <p>The challenges are really very simple. Network security, keeping intruders off of your network, was the most widely used topic and grew 5%. Firewalls, which are an important component of network security, grew 16%. Hardening, a much smaller topic that addresses making systems less vulnerable to attack, grew 110%. Penetration testing remained one of the most widely used topics. Usage dropped 5%, although a 10% increase for Kali Linux (an important tool for penetration testers) largely offsets that decline.</p> <p>The 22% growth in security governance is another indicator of changed attitudes: security is no longer an ad hoc exercise that waits for something to happen and then fights fires. Security requires planning, training, testing, and auditing to ensure that policies are effective.</p> <p>One key to security is knowing who your users are and which parts of the system each user can access. Identity and access management (IAM) has often been identified as a weakness, particularly for cloud security. As systems grow more complex, and as our concept of “identity” evolves from individuals to roles assigned to software services, IAM becomes much more than usernames and passwords. It requires a thorough understanding of who the actors are on your systems and what they’re allowed to do. This extends the old idea of “least privilege”: each actor needs the ability to do exactly what they need, no more and no less. The use of content about IAM grew 8.0% in the past year. It’s a smaller gain than we would have liked to see but not insignificant.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig6-710x1048.png" alt="" class="wp-image-15369" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig6-710x1048.png 710w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig6-203x300.png 203w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig6-768x1134.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig6-1040x1536.png 1040w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig6.png 1220w" sizes="(max-width: 710px) 100vw, 710px" /><figcaption>Figure 6. Security</figcaption></figure> <p>Application security grew 42%, showing that software developers and operations staff are getting the message. The DevSecOps “shift left” movement, which focuses on software security early in the development process, appears to be winning; use of content about DevSecOps was up 30%. Similarly, those who deploy and maintain applications have become even more aware of their responsibilities. Developers may design identity and access management into the code, but operations is responsible for configuring these correctly and ensuring that access to applications is only granted appropriately. Security can’t be added after the fact; it has to be part of the software process from beginning to the end.</p> <p>Advanced persistent threats (APTs) were all over the news a few years ago. We don’t see the term APT anywhere near as much as we used to, so we’re not surprised that usage has dropped by 35%. Nevertheless, nation-states with sophisticated offensive capabilities are very real, and cyber warfare is an important component of several international conflicts, including the war in Ukraine.</p> <p>It’s disappointing to see that usage of content about zero trust has declined by 20%. That decrease is more than offset by the increase in IAM, which is an essential tool for zero trust. But don’t forget that IAM is just a tool and that the goal is to build systems that don’t rely on trust, that always verify that every actor is appropriately identified and authorized. How can you defend your IT infrastructure if you assume that attackers already have access? That’s the question zero trust answers. Trust nothing; verify everything.</p> <p>Finally, compliance is down 27%. That’s more than offset by the substantial increase of interest in governance. Auditing for compliance is certainly a part of governance. Focusing on compliance itself, without taking into account the larger picture, is a problem rather than a solution. We’ve seen many companies that focus on compliance with existing standards and regulations while avoiding the hard work of analyzing risk and developing effective policies for security. “It isn’t our fault that something bad happened; we followed all the rules” is, at best, a poor way to explain systemic failure. If that compliance-oriented mindset is fading, good riddance. Compliance, understood properly, is an important component of IT governance. Understood badly, compliance is an unacceptable excuse.</p> <p>Finally, a word about a topic that doesn’t yet appear in our data. There has, of course, been a lot of chatter about the use of AI in security applications. AI will be a great asset for log file analysis, intrusion detection, incident response, digital forensics, and other aspects of cybersecurity. But, as we’ve already said, there are always two sides to AI. How does AI change security itself? Any organization with AI applications will have to protect them from exploitation. What vulnerabilities does AI introduce that didn’t exist a few years ago? There are many articles about prompt injection, sneaky prompts designed to “jailbreak” AI systems, data leakage, and other vulnerabilities—and we believe that’s only the beginning. Securing AI systems will be a critical topic in the coming years.</p> <h2>Cloud Computing</h2> <p>Looking at platform usage for cloud-related topics, one thing stands out: cloud native. Not only is it the most widely used topic in 2023, but it grew 175% from 2022 to 2023. This marks a real transition. In the past, companies built software to run on-premises and then moved it to the cloud as necessary. Despite reports (<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://learning.oreilly.com/library/view/the-cloud-in/9781492096733/ch01.html#executive_summary" target="_blank">including ours</a>) that showed 90% or more “cloud adoption,” we always felt that was very optimistic. Sure, 90% of all companies may have one or two experiments <em>in</em> the cloud—but are they really building <em>for</em> the cloud? This huge surge in cloud native development shows that we’ve now crossed that chasm and that companies have stopped kicking the tires. They’re building for the cloud as their primary deployment platform.</p> <p>You could, of course, draw the opposite conclusion by looking at cloud deployment, which is down 27%. If companies are developing for the cloud, how are those applications being deployed? That’s a fair question. However, as cloud usage grows, so does organizational knowledge of cloud-related topics, particularly deployment. Once an IT group has deployed its first application, the second isn’t necessarily “easy” or “the same,” but it is familiar. At this point in the history of cloud computing, we’re seeing few complete newcomers. Instead we’re seeing existing cloud users deploying more and more applications. We’re also seeing a rise in tools that streamline cloud deployment. Indeed, any provider worth thinking about has a tremendous interest in making deployment as simple as possible.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig7-1048x996.png" alt="" class="wp-image-15370" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig7-1048x996.png 1048w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig7-300x285.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig7-768x730.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig7.png 1209w" sizes="(max-width: 1048px) 100vw, 1048px" /><figcaption>Figure 7. Cloud architecture</figcaption></figure> <p>Use of content about cloud security grew 25%, and identity and access management (IAM) grew 8%. An epidemic of data theft and ransomware that continues to this day put security on the corporate map as a priority, not just an expense with annual budget requests that sounded like an extortion scam: “Nothing bad happened this year; give us more money and maybe nothing bad will happen next year.” And while the foundation of any security policy is good local security hygiene, it’s also true that the cloud presents its own issues. Identity and access management: locally, that means passwords, key cards, and (probably) two-factor authentication. In the cloud, that means IAM, along with zero trust. Same idea, but it would be irresponsible to think that these aren’t more difficult in the cloud.</p> <p>Hybrid cloud is a smaller topic area that has grown significantly in the past year (145%). This growth points partly to the cloud becoming the de facto deployment platform for enterprise applications. It also acknowledges the reality of how cloud computing is adopted. Years ago, when “the cloud” was getting started, it was easy for a few developers in R&D to expense a few hours of time on AWS rather than requisitioning new hardware. The same was true for data-aware marketers who wanted to analyze what was happening with their potential customers—and they might choose Azure. When senior management finally awoke to the need for a “cloud strategy,” they were already in a hybrid situation, with multiple wildcat projects in multiple clouds. Mergers and buyouts complicated the situation more. If company A is primarily using AWS and company B has invested heavily in Google Cloud, what happens when they merge? Unifying behind a single cloud provider isn’t going to be worth it, even though cloud providers are providing tools to simplify migration (at the same time as they make their own clouds difficult to leave). The cloud is naturally hybrid. “Private cloud” and “public cloud,” when positioned as alternatives to each other and to a hybrid cloud, smell like “last year’s news.” It’s not surprising that usage has dropped 46% and 10%, respectively.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig8-1048x759.png" alt="" class="wp-image-15371" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig8-1048x759.png 1048w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig8-300x217.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig8-768x556.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig8.png 1209w" sizes="(max-width: 1048px) 100vw, 1048px" /><figcaption>Figure 8. Cloud providers</figcaption></figure> <p>What about the perennial horse race between Amazon Web Services, Microsoft Azure, and Google Cloud? Is anyone still interested, except perhaps investors and analysts? AWS showed a very, very small gain (0.65%), but Azure and Google Cloud showed significant losses (16% and 22%, respectively). We expected to see Azure catch up to AWS because of its lead in AI as a service, but it didn’t. As far as our platform is concerned, that’s still in the future.</p> <h2>Web Development</h2> <p>React and Angular continue to dominate web development. JavaScript is still the lingua franca of web development, and that isn’t likely to change any time soon.</p> <p>But the usage pattern has changed slightly. Last year, React was up, and Angular was sharply down. This year, usage of React content hasn’t changed substantially (down 0.33%). Angular is down 12%, a smaller decline than last year but still significant. When a platform is as dominant as React, it may have nowhere to go but down. Is momentum shifting?</p> <p>We see some interesting changes among the less popular frameworks, both old and new. First, Vue isn’t a large part of the overall picture, and it isn’t new—it’s been around since 2014—but if its 28% annual growth continues, it will soon become a dominant framework. That increase represents a solid turnaround after losing 17% from 2021 to 2022. Django is even older (created in 2005), but it’s still widely used—and with an 8% increase this year, it’s not going away. FastAPI is the newest of this group (2018). Even though it accounts for a very small percentage of platform use, it’s easy for a small change in usage to have a big effect. An 80% increase is hard to ignore.</p> <p>It’s worth looking at these frameworks in a little more detail. Django and FastAPI are both Python-based, and FastAPI takes full advantage of Python’s type hinting feature. Python has long been an also-ran in web development, which has been dominated by JavaScript, React, and Angular. Could that be changing? It’s hard to say, and it’s worth noting that Flask, another Python framework, showed a 12% decrease. As a whole, Python frameworks probably declined from 2022 to 2023, but that may not be the end of the story. Given the number of boot camps training new web programmers in React, the JavaScript hegemony will be hard to overcome.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig9-702x1048.png" alt="" class="wp-image-15372" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig9-702x1048.png 702w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig9-201x300.png 201w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig9-768x1147.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig9-1028x1536.png 1028w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig9.png 1209w" sizes="(max-width: 702px) 100vw, 702px" /><figcaption>Figure 9. Web development</figcaption></figure> <p>What about PHP, another long-standing framework that dates back to 1995, when the web was indeed young? PHP grew 5.9% in the past year. The use of content about PHP is small compared to frameworks like React and Angular or even Django. PHP certainly doesn’t inspire the excitement that it did in the 1990s. But remember that over 80% of the web is built on PHP. It’s certainly not trendy, it’s not capable of building the feature-rich sites that many users expect—but it’s everywhere. WordPress (down 4.8%), a content management system used for millions of websites, is based on PHP. But regardless of the number of sites that are built on PHP or WordPress, Indeed shows roughly <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="http://indeed.com/" target="_blank">three times as many job openings for React developers</a> as for PHP and WordPress combined. PHP certainly isn’t going away, and it may even be growing slightly. But we suspect that PHP programmers spend most of their time maintaining older sites. They already know what they need to do that, and neither of those factors drives content usage.</p> <p>What about some other highly buzzworthy technologies? After showing 74% growth from 2021 to 2022, WebAssembly (Wasm) declined by 41% in 2023. Blazor, a web framework for C# that generates code for Wasm, declined by 11%. Does that mean that Wasm is dying? We still believe Wasm is a very important technology, and we frequently read about amazing projects that are built with it. It isn’t yet a mature technology—and there are plenty of developers willing to argue that there’s no need for it. We may disagree, but that misses the point. Usage of Wasm content will probably decline gradually…until someone creates a killer application with it. Will that happen? Probably, but we can’t guess when.</p> <p>What does this mean for someone who’s trying to develop their skills as a web developer? First, you still can’t go wrong with React, or even with Angular. The other JavaScript frameworks, such as Next.js, are also good options. Many of these are metaframeworks built on React, so knowing them makes you more versatile while leveraging knowledge you already have. If you’re looking to broaden your skills, Django would be a worthwhile addition. It’s a very capable framework, and knowing Python will open up other possibilities in software development that may be helpful in the future, even if not now.</p> <h2>Certification</h2> <p>This year, we took a different approach to certification. Rather than discussing certification for different subject areas separately (that is, cloud certification, security certification, etc.), we used data from the platform to build a list of the top 20 certifications and grouped them together. That process gives a slightly different picture of which certifications are important and why. We also took a brief look at O’Reilly’s new badges program, which gives another perspective on what our customers want to learn.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig10-658x1048.png" alt="" class="wp-image-15373" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig10-658x1048.png 658w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig10-188x300.png 188w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig10-768x1223.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig10-965x1536.png 965w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig10.png 1213w" sizes="(max-width: 658px) 100vw, 658px" /><figcaption>Figure 10. Certification</figcaption></figure> <p>Based on the usage of content in our platform (including practice tests), the most popular certifications are security certifications: CISSP (which declined 4.8%) and CompTIA Security+ (which grew 6.0%). CISSP is an in-depth exam for security professionals, requiring at least five years’ experience before taking the exam. Security+ is more of an entry-level exam, and its growth shows that security staff are still in demand. ISACA’s Certified Information Security Manager (CISM) exam, which focuses on risk assessment, governance, and incident response, isn’t as popular but showed a 54% increase. CompTIA’s Certified Advanced Security Practitioner (CASP+) showed a 10% increase—not as large but part of the same trend. The Certified Ethical Hacker (CEH) exam, which focuses on techniques useful for penetration testing or red-teaming, is up 4.1%, after a decline last year. Those increases reflect where management is investing. Hoping that there won’t be an incident has been replaced by understanding exposure, putting in place governance mechanisms to minimize risk, and being able to respond to incidents when they occur.</p> <p>What really stands out, however, isn’t security: it’s the increased use of content about CompTIA A+, which is up 58%. A+ isn’t a security exam; it’s advertised as an entry-level exam for IT support, stressing topics like operating systems, managing SaaS for remote work, troubleshooting software, hardware, and networking problems, and the like. It’s testimony to the large number of people who want to get into IT. Usage of content about the CompTIA Linux+ exam was much lower but also grew sharply (23%)—and, as we’ve said in the past, Linux is “table stakes” for almost any job in computing. It’s more likely that you’ll encounter Linux indirectly via containers or cloud providers rather than managing racks of computers running Linux; but you will be expected to know it. The Certified Kubernetes Administrator (CKAD) exam also showed significant growth (32%). Since it was first released in 2014, Kubernetes has become an inescapable part of IT operations. The biggest trend in IT, going back 70 years or so, has been the increase in the ratio of machines to operators: from multiple operators per machine in the ’60s to one operator per machine in the era of minicomputers to dozens and now, in the cloud, to hundreds and thousands. Complex as Kubernetes is—and we admit, we keep looking for a simpler alternative—it’s what lets IT groups manage large applications that are implemented as dozens of microservices and that run in thousands of containers on an uncountable number of virtual machines. Kubernetes has become an essential skill for IT. And certification is becoming increasingly attractive to people working in the field; there’s no other area in which we see so much growth.</p> <p>Cloud certifications also show prominently. Although “the cloud” has been around for almost 20 years, and almost every company will say that they are “in the cloud,” in reality many companies are still making that transition. Furthermore, cloud providers are constantly adding new services; it’s a field where keeping up with change is difficult. Content about Amazon Web Services was most widely used. AWS Cloud Practitioner increased by 35%, followed by AWS Solutions Architect (Associate), which increased 15%. Microsoft Azure certification content followed, though the two most prominent exams showed a decline: Azure Fundamentals (AZ-900) was down 37%, and Azure Administration (AZ-104) was down 28%. Google Cloud certifications trailed the rest: Google’s Cloud Engineer showed solid growth (14%), while its Data Engineer showed a significant decline (40%).</p> <p>Content about Microsoft’s AI-900 exam (Azure AI Fundamentals) was the least-used among the certifications that we tracked. However, it gained 121%—it more than doubled—from 2022 to 2023. While we can’t predict next year, this is the sort of change that trends are made of. Why did this exam suddenly get so hot? It’s easy, really: Microsoft’s investment in OpenAI, its integration of the GPT models into Bing and other products, and its AI-as-a-service offerings through Azure have suddenly made the company a leader in cloud-based AI. While we normally hedge our bets on smaller topics with big annual growth—it’s easy for a single new course or book to cause a large swing—AI isn’t going away, nor is Microsoft’s leadership in cloud services for AI developers.</p> <p>Late in 2023, O’Reilly began to offer <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/online-learning/badges.html" target="_blank">badges tied to course completion</a> on the O’Reilly learning platform. Badges aren’t certifications, but looking at the top badges gives another take on what our customers are interested in learning. The results aren’t surprising: Python, GPT (not just ChatGPT), Kubernetes, software architecture, and Java are the most popular badges.</p> <p>However, it’s interesting to look at the difference between our B2C customers (customers who have bought platform subscriptions as individuals) and B2B customers (who use the platform via a corporate subscription). For most topics, including those listed above, the ratio of B2B to B2C customers is in the range of 2:1 or 3:1 (two or three times as many corporate customers as individuals). The outliers are for topics like communications skills, Agile, Scrum, personal productivity, Excel, and presentation skills: users from B2B accounts obtained these badges four (or more) times as often as users with personal accounts. This makes sense: these topics are about teamwork and other skills that are valuable in a corporate environment.</p> <p>There are few (if any) badge topics for which individual (B2C) users outnumbered corporate customers; that’s just a reflection of our customer base. However, there were some topics where the ratio of B2B to B2C customers was closer to one. The most interesting of these concerned artificial intelligence: large language models (LLMs), TensorFlow, natural language processing, LangChain, and MLOps. Why is there more interest among individuals than among corporate customers? Perhaps by next year we’ll know.</p> <h2>Design</h2> <p>The important story in design is about tools. Topics like user experience and web design are stable or slightly down (down 0.62% and 3.5%, respectively). But usage about design tools is up 105%, and the VC unicorn Figma is up 145%. Triple-digit growth probably won’t continue, but it’s certainly worth noticing. It highlights two important trends that go beyond typical design topics, like UX.</p> <p>First, low-code and no-code tools aren’t new, but many new ones have appeared in the past year. Their success has been aided by artificial intelligence. We already have AI tools that can generate text, whether for a production site or for a mockup. Soon we’ll have no-code tools that don’t just spit out a wireframe but will be able to implement the design itself. They will be smart about what the user wants them to do. But to understand the importance of low-code to design, you have to look beyond the use designers will make of these tools. Designers will also be designing these tools, along with other AI-powered applications. Tools for designers have to be well-designed, of course: that’s trivial. But what many discussions about AI ignore is that designing applications that use AI well is far from trivial. We’ve all been blindsided by the success of ChatGPT, which made the GPT models instantly accessible to everyone. But once you start thinking about the possibilities, you realize that a chat is hardly an ideal interface for an AI system.<sup>2</sup> What will the users of these systems really need? We’ve only just started down that path. It will be an exciting journey—particularly for designers.</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig11-838x1048.png" alt="" class="wp-image-15374" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig11-838x1048.png 838w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig11-240x300.png 240w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig11-768x960.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig11.png 1209w" sizes="(max-width: 838px) 100vw, 838px" /><figcaption>Figure 11. Design</figcaption></figure> <p>Second, Figma is important because it’s a breakthrough in tools for collaboration. Tools that allow remote employees to collaborate productively are crucial when coworkers can be anywhere: in an office, at home, or on another continent. The last year and a half has been full of talk about virtual reality, metaverses, and the like. But what few have realized is that the metaverse isn’t about wearing goggles—it’s about seamless collaboration with friends and coworkers. Use of content about AR and VR dropped 25% because people have missed the real story: we don’t need 3D goggles; we need tools for collaboration. And, as with low-code, collaboration tools are both something to design with and something that needs to be designed. We’re on the edge of a new way to look at the world.</p> <p>Use of content about information architecture was up 16%, recovering from its decline from 2021 to 2022. The need to present information well, to design the environments in which we consume information online, has never been more important. Every day, there’s more information to absorb and to navigate—and while artificial intelligence will no doubt help with that navigation, AI is as much a design problem as a design solution. (Though it’s a “good problem” to have.) Designing and building for accessibility is clearly related to information architecture, and it’s good to see more engagement with that content (up 47%). It’s been a long time coming, and while there’s still a long way to go, accessibility is being taken more seriously now than in the past. Websites that are designed to be usable by people with impairments aren’t yet the rule, but they’re no longer exceptions.</p> <h2>Professional Development</h2> <p>Almost everyone involved with software starts as a programmer. But that’s rarely where they end. At some point in their career, they are asked to write a specification, lead a team, manage a group, or maybe even found a company or serve as an executive in an existing company.</p> <p>O’Reilly is the last company to believe that software developers are neck-bearded geeks who want nothing more than to live in a cave and type on their terminals. We’ve spent most of our history fighting against that stereotype. Nevertheless, going beyond software development is a frequent source of anxiety. That’s no doubt true for anyone stepping outside their comfort zone in just about any field, whether it’s accounting, law, medicine, or something else. But at some point in your career, you have to do something that you aren’t prepared to do. And, honestly, the best leaders are usually the ones who have some anxiety, not the ones whose reaction is “I was born to be a leader.”</p> <figure class="wp-block-image size-large"><img src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig12-1048x762.png" alt="" class="wp-image-15375" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig12-1048x762.png 1048w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig12-300x218.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig12-768x558.png 768w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Fig12.png 1209w" sizes="(max-width: 1048px) 100vw, 1048px" /><figcaption>Figure 12. Professional development</figcaption></figure> <p>For the past few years, our audience has been interested in professional growth that goes beyond just writing software or building models for AI and ML. Project management is up 13%; the ability to manage large projects is clearly seen as an asset for employees who are looking for their next promotion (or, in some cases, their next job). Whatever their goals might be, anyone looking for a promotion or a new job—or even just solidifying their hold on their current job—would be well served by improving their communications skills (up 23%). Professional development (up 22%) is a catch-all topic that appears to be responding to the same needs. What’s driving this? 2023 began and ended with a lot of news about layoffs. But despite well-publicized layoffs from huge companies that overhired during the pandemic, there’s little evidence that the industry as a whole has suffered. People who are laid off seem to be snapped up quickly by new employers. Nevertheless, anxiety is real, and the emphasis we’re seeing on professional development (and specifically, communications and project management skills) is partially a result of that anxiety. Another part of the story is no doubt <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/fearing-the-wrong-thing/" target="_blank">the way AI is changing the workplace</a>. If generative AI makes people more efficient, it frees up time for them to do other things, including strategic thinking about product development and leadership. It may finally be time to value “individuals and interactions over processes and tools,” and “customer collaboration over contract negotiation,” as the <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://agilemanifesto.org/" target="_blank"><em>Agile Manifesto</em></a> claims. Doing so will require a certain amount of reeducation, focusing on areas like communications, interpersonal skills, and strategic thinking.</p> <p>Product management, the discipline of managing a product’s lifecycle from the initial idea through development and release to the market, is also a desirable skill. So why is it only up 2.8% and not 20% like project management? Product management is a newer position in most companies; it has strong ties to marketing and sales, and as far as fear of layoffs is concerned (whether real or media driven), product management positions may be perceived as more vulnerable.</p> <p>A look at the bottom of the chart shows that usage of content that teaches critical thinking grew 39%. That could be in part a consequence of ChatGPT and the explosion in artificial intelligence. Everyone knows that AI systems make mistakes, and almost every article that discusses these mistakes talks about the need for critical thinking to analyze AI’s output and find errors. Is that the cause? Or is the desire for better critical thinking skills just another aspect of professional growth?</p> <h2>A Strange Year?</h2> <p>Back at the start, I said this was a strange year. As much as we like to talk about the speed at which technology moves, reality usually doesn’t move that fast. When did we first start talking about data? Tim O’Reilly said “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/pub/a/web2/archive/what-is-web-20.html?page=3" target="_blank">Data is the next Intel Inside</a>” in 2005, almost 20 years ago. Kubernetes has been around for a decade, and that’s not counting its prehistory as Google’s Borg. Java was introduced in 1995, almost 30 years ago, and that’s not counting its set-top box prehistory as Oak and Green. C++ first appeared in 1985. Artificial intelligence has a prehistory as long as computing itself. When did AI emerge from its wintry cave to dominate the data science landscape? 2016 or 2017, when we were amazed by programs that could sort images into dogs and cats? Sure, Java has changed a lot; so has what we do with data. Still, there’s more continuity than disruption.</p> <p>This year was one of the few years that could genuinely be called disruptive. Generative AI will change this industry in important ways. Programmers won’t become obsolete, but programming as we know it might. Programming will have more to do with understanding problems and designing good solutions than specifying, step-by-step, what a computer needs to do. We’re not there yet, but we can certainly imagine a day when a human language description leads reliably to working code, when “Do what I meant, not what I said” ceases to be the programmer’s curse. That change has already begun, with tools like GitHub Copilot. But to thrive in that new industry, programmers will need to know more about architecture, more about design, more about human relations—and we’re only starting to see that in our data, primarily for topics like product management and communications skills. And perhaps that’s the definition of “disruptive”: when our systems and our expectations change faster than our ability to keep up. I’m not worried about programmers “losing their jobs to an AI,” and I really don’t see that concern among the many programmers I talk to. But whatever profession you’re in, you will lose out if you don’t keep up. That isn’t kind or humane; that’s capitalism. And perhaps I should have used ChatGPT to write this report.<sup>3</sup></p> <p>Jerry Lee Lewis might have said “There’s a whole lotta disruption goin’ on.” But despite all this disruption, much of the industry remains unchanged. People seem to have tired of the terms DevOps and SRE, but so it goes: the half-life of a buzzword is inevitably short, and these have been extraordinarily long-lived. The problems these buzzwords represent haven’t gone away. Although we aren’t yet collecting the data (and don’t yet have enough content for which to collect data), developer platforms, self-service deployment, and platform engineering look like the next step in the evolution of IT operations. Will AI play a role in platform engineering? We’d be surprised if it didn’t.</p> <p>Movement to the cloud continues. While we’ve heard talk of cloud “repatriation,” we see no evidence that it’s happening. We do see evidence that organizations realize that the cloud is naturally hybrid and that focusing on a single cloud provider is short-sighted. There’s also evidence that organizations are now paying more than lip service to security, particularly cloud security. That’s a very good sign, especially after many years in which companies approached security by hoping nothing bad would happen. As many chess grandmasters have said, “Hope is never a good strategy.”</p> <p>In the coming year, AI’s disruption will continue to play out. What consequences will it have for programming? How will jobs (and job prospects) change? How will IT adapt to the challenge of managing AI applications? Will they rely on AI-as-a-service providers like OpenAI, Azure, and Google, or will they build on open source models, which will probably run in the cloud? What new vulnerabilities will AI applications introduce into the security landscape? Will we see new architectural patterns and styles? Will AI tools for software architecture and design help developers grapple with the difficulties of microservices, or will it just create confusion?</p> <p>In 2024, we’ll face all of these questions. Perhaps we’ll start to see answers. One thing is clear: it’s going to be an exciting year.</p> <hr class="wp-block-separator" /> <h3>Footnotes</h3> <ol><li>Google Trends <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://trends.google.com/trends/explore?geo=US&q=chatgpt&hl=en" target="_blank">suggests</a> that we may be seeing a resurgence in ChatGPT searches. Meanwhile, searches for ChatGPT on our platform appear to have bottomed out in October, with a very slight increase in November. This discrepancy aligns well with the difference between our platform and Google’s. If you want to use ChatGPT to write a term paper, are you going to search Google or O’Reilly?</li><li>Phillip Carter’s article, “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm" target="_blank">All the Hard Stuff Nobody Talks About when Building Products with LLMs</a>,” is worth reading. While it isn’t specifically about design, almost everything he discusses is something designers should think about.</li><li>I didn’t. Not even for data analysis. </li></ol>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/technology-trends-for-2024/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>I Actually Chatted with ChatGPT</title> <link>https://www.oreilly.com/radar/i-actually-chatted-with-chatgpt/</link> <comments>https://www.oreilly.com/radar/i-actually-chatted-with-chatgpt/#respond</comments> <pubDate>Tue, 16 Jan 2024 10:52:10 +0000</pubDate> <dc:creator><![CDATA[Philip Guo]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Deep Dive]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15329</guid> <description><![CDATA[ChatGPT was released just over a year ago (at the end of November 2022), and countless people have already written about their experiences using it in all sorts of settings. (I even contributed my own hot take last year with my O’Reilly Radar article Real-Real-World Programming with ChatGPT.) What more is left to say by […]]]></description> <content:encoded><![CDATA[<p>ChatGPT was released just over a year ago (at the end of November 2022), and countless people have already written about their experiences using it in all sorts of settings. (I even contributed my own hot take last year with my O’Reilly Radar article <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.oreilly.com/radar/real-real-world-programming-with-chatgpt/" target="_blank"><em>Real-Real-World Programming with ChatGPT</em></a>.) What more is left to say by now? Well, I bet very few of those people have actually <em>chatted</em> with ChatGPT. And by “chat” I mean the original sense of the word—to hold a back-and-forth verbal conversation with it just like how you would chat with a fellow human being. I recently chatted with ChatGPT, and I want to use that experience to reflect on the usability of voice interfaces for AI tools based on Large Language Models. I’m personally interested in this topic since I am a professor who <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://pg.ucsd.edu/" target="_blank">researches human-computer interaction, user experience design, and cognitive science</a>, so AI voice interfaces are fascinating to me.</p> <p>Here’s what I did: In December 2023 I installed the <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openai.com/blog/introducing-the-chatgpt-app-for-ios" target="_blank">official ChatGPT iOS app from OpenAI</a> on my iPhone and used its voice input mode to hold several hour-long conversations with it while driving long-distance on California highways. I wore standard Apple earbuds with a built-in mic and talked with ChatGPT just like how I would be talking to someone on the phone while driving. These long solo drives were the perfect opportunity to test out ChatGPT’s voice feature because I couldn’t interact with the app using my hands for safety reasons.</p> <p>I had a very clear use case in mind: <strong>I wanted a conversation partner to keep me awake and alert while driving long-distance by myself.</strong> I’ve found that listening to music or podcasts doesn’t keep me alert when I’m tired because it’s such a passive experience—but what does keep me awake is having someone to talk to, either in the car or remotely on the phone. Could ChatGPT replace a human conversation partner in this role?</p> <h3><strong>The Good: ChatGPT Made Personalized Podcasts to Keep Me Engaged While Driving</strong></h3> <p>To not bury the lede, it turns out that it did a remarkable job! As I was driving I was able to engage in several hour-long conversations with ChatGPT that ended only because I had to take a rest stop or hit the usage limit for GPT-4. (I pay for a ChatGPT Plus subscription so I can use the most advanced GPT-4 model, but that comes with a usage limit that I usually hit after about an hour.)</p> <p>The best way to describe my experience is (borrowing a wonderful term my friend coined) that it felt like listening to a <em>personalized podcast</em>. Since ChatGPT did most of the talking, it was a mostly passive listening experience on my part except for times when I wanted to ask follow-up questions or direct it to change topics. Critically, this meant I could still focus most of my attention on driving safely with a level of distraction on par with listening to a podcast. But it kept me more alert than a regular podcast since I could actively direct the flow of the conversation.</p> <p>For a concrete example of what such a personalized podcast felt like, I started one conversation by straight-up asking ChatGPT to keep me awake while I was driving in Southern California from Los Angeles to San Diego. So it started by making small talk about road trips in general and asking me about various California landmarks that I’ve visited, culminating in asking me more about San Diego (where I live). When it asked me what places I liked visiting the most here, I mentioned the San Diego Zoo and it started telling me a bit about what makes this particular zoo notable. It mentioned the concept of “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.avma.org/javma-news/2002-12-01/designing-zoo-habitats-promote-animal-well-being" target="_blank">naturalistic enclosures</a>”—a term I had not heard before—so I asked it to elaborate on what this meant. ChatGPT’s explanation of this concept got me interested in the history of zoos, especially the progression from keeping animals in cages to today’s cageless naturalistic enclosures, which aim to be better for animal welfare. During that segment it mentioned the term “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Menagerie" target="_blank">menagerie</a>” in passing, which I had not heard of in that context before, so I asked it to elaborate more. It then went back farther in history to describe how a menagerie refers to the phenomenon of ancient rulers keeping exotic animals for display without as much regard for the animals’ well-being. Listening to that made me realize that I had actually heard the term menagerie in reference to a <em>Star Trek</em> episode of some sort, but I forgot which one, so I asked ChatGPT to jog my memory. It turns out that “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/The_Menagerie_(Star_Trek:_The_Original_Series)" target="_blank">The Menagerie</a>” was a very famous episode of the original <em>Star Trek</em> television series, so after chatting about that episode and other famous <em>Star Trek</em> episodes for a bit, we got onto the topic of why that show was canceled after only three seasons but later found a much larger audience in syndication (i.e., reruns). That in turn got me curious about the concept of syndication in the television business, so ChatGPT dived more into this topic. A few more conversational twists and turns later, then I suddenly realized that the hour had flown by and it was time to pull over for a bathroom break. Success!</p> <p>Now, I don’t expect you to care at all about the details of the conversation I just described since it wasn’t your conversation—it was mine! But I certainly cared about it at the time since I was genuinely curious to learn more about the topics that ChatGPT mentioned, often offhand in the midst of telling me about something else. It felt a bit like diving down a Wikipedia rabbit hole of following related links, where each follow-up question I asked led it down another meandering path. It was perfect for keeping me from getting bored and sleepy during my long drive.</p> <p>ChatGPT isn’t just good at this sort of superficial “personalized podcast about Wikipedia-level trivia” … it could also engage me in a more substantive conversation about a task I actually needed help with at the moment. In another hour-long car chat, I prompted ChatGPT to help me design a method to organize my huge collection of almost 30 years’ worth of personal and work-related files for backup. I’ve been diligent about data backup throughout my life, but my files are fragmented amongst different media over the years—burning CDs and DVDs back in the day, several generations of external hard drives (that are in various states of decay), university servers, Dropbox, and other cloud services. For years I had an aspirational goal of unifying all of my backups into one central directory tree, akin to the concept of a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Monorepo" target="_blank">monorepo</a> in software development. I’ve recently been brainstorming ideas for how to design such a system and how to deal with the practical challenges of scaling and maintenance. So I figured that ChatGPT could help me brainstorm during one of my long drives. Again it did a good job at engaging me in this bespoke conversation, and the hour flew by before I had to take a rest stop. I won’t bore you with details of what we discussed, but it felt like talking with an expert in data management who was giving me advice about how to deal with my particular challenge.</p> <h3><strong>Intermission: Why It Feels Kind of Magical</strong></h3> <p>Skeptical readers may be thinking at this point, “What’s the big deal, it’s just ChatGPT under the hood. I can already do all this from my computer by typing into the ChatGPT text box!” Although that’s technically true, there’s something magical about being able to do this all hands-free via voice. If you don’t believe me, just try it for an hour. My folk theory is that speaking and listening are hardwired into our brain’s innate language circuitry, but writing and reading are learned skills (i.e., “software” rather than “hardware” in our brains). And that’s why it feels more magical to hold a verbal conversation with an AI versus having the exact same conversation in a text box on a screen. If the AI is good enough, then it almost feels like you’re talking to a real person … at certain times when I was getting deep into a back-and-forth conversation I nearly forgot I was talking to a machine. However, that illusion broke in several ways …</p> <h3><strong>The Not-So-Good: Usability Limitations of the ChatGPT Voice Interface</strong></h3> <p>Despite my positive experiences with ChatGPT’s voice mode, it still didn’t live up to the gold standard of feeling like I was talking with a fellow human being. That’s okay, though, since this is an incredibly high bar! Here are some of the ways it fell short.</p> <ul><li><strong>Must speak entire request all at once</strong>: Most notably, it felt unnatural to have to speak my entire request all at once without pausing. Whenever I paused for too long, ChatGPT would interpret what I said so far as my request and start processing it. As an analogy, when typing a request in a text chat, you can hit the Enter or Send buttons … imagine how weird it would be if ChatGPT started answering you the very moment you stopped typing for one second! Note that in human conversations, especially face-to-face, we use visual cues to tell whether our conversation partner is done talking or whether they are pausing a bit to think about the next thing to say. Even over the phone, we can tell by vocal inflections whether they are temporarily paused and want to keep talking, or whether they are done with their turn and ready for us to respond. Since ChatGPT can’t do any of that (yet!) I often had to think hard about what I wanted to say and then say it all at once without pausing. This was fine for simple requests like “Tell me more about naturalistic enclosures in zoos,” but for more complex requests like describing some facet of my data backup setup, it was painful to have to blurt out as much as I could without pausing. Even more annoyingly, I would sometimes make mistakes when talking so much all at once without pausing. Ideally the app would do a better job at detecting pauses in human speech, taking both context and vocal intonations into account. An easier hack would be to have a voice command like “DONE” or “OVER” (like when people use walkie-talkies) to signal that I am done talking; however, this would also feel unnatural for casual users.</li><li><strong>Unpredictable wait times</strong>: Wait times (latency) for ChatGPT’s responses are unpredictable, and there aren’t audio cues to help me establish an expectation for how long I need to wait before it responds. There’s a click sound when it starts processing my request, but then I may need to wait a few seconds in silence before hearing a response … maybe it’s only one second or maybe it’s five seconds. That said, if I ask it to browse the web, then it plays a continuous waiting sound; web browsing takes longer, maybe 10 to 20 seconds, but at least I get to hear a “waiting” sound. (I don’t mind ChatGPT taking longer here since a human would also take more time to browse the web. However, web browsing is annoying when I don’t explicitly ask it to browse. Oftentimes I want a fast answer but something I say triggers a browse without me intending to.) In contrast, when speaking with a human face-to-face, I can use visual cues to tell whether the other person is deep in thought or when they will likely respond; and even over the phone the other person may say “ummm” or “hold on one sec, lemme think” or “ok let me look this up on the web, hang tight for a while …” if they need more time to think through their response. However, since I don’t get any of these verbal cues from ChatGPT, unpredictable wait times break the illusion of talking to a person.</li><li><strong>Cannot interrupt while it is speaking</strong>: I always had to wait for ChatGPT to completely finish talking before it would listen to my next request. And since I never know ahead of time how long it planned to talk for during a particular turn (i.e., how many words its LLM-generated response is), when I wanted to say something midway it was aggravating to have to wait. I later saw that I could actually interrupt it by tapping on the app on my phone screen, but since I was driving and hands-free, I couldn’t safely do that. Also, that seems like a cumbersome interaction; I should be able to just talk when I want to, even when it is talking. This limitation made the conversation feel like we were using a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://en.wikipedia.org/wiki/Walkie-talkie" target="_blank">walkie-talkie</a> where only one party can talk at once. And it’s not just me—this concept of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=overlapping+speech&btnG=" target="_blank">overlapping speech</a> is widely studied in linguistics and communication research. Humans naturally talk over one another for various reasons, so not being able to do this with ChatGPT made our conversation feel less fluid. Even implementing a feature like a voice command for interruption would be great, like maybe if I say “pause” or “wait” then it could stop and await my request.</li><li><strong>Speech recognition errors</strong>: ChatGPT’s speech recognition system (presumably based on <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/openai/whisper" target="_blank">OpenAI’s open source Whisper model</a>) is very good, but it does at times misinterpret what I’m saying. What’s stranger is that sometimes it thinks I said something when I didn’t, maybe because it picked up on background rumbles in my car. Several times I wouldn’t be saying anything and suddenly it responds out of the blue; and when I check the written transcript later, it thinks that I said something like “Thank you for watching!” (which I never said). At other times it tries to prematurely end the conversation even though I’m not done, maybe because it mistakenly detected that I said something along the lines of “Thanks …” without any follow-up. Misrecognizing words is forgivable, but I feel that it shouldn’t ever interpret background sounds as words. Of course, if there were other people in the car with me and either they talked or I was talking to them, then I could also understand how ChatGPT would mistakenly interpret that as being a request for it; always-listening home assistants like Alexa have had this issue for years. A more advanced AI would learn to filter out both other people’s voices and also infer when I was speaking with someone else and not it. For instance, when it detects that my sentence is way off topic, maybe that means I’m speaking with someone else in the car; it could at least ask me “Were you talking to me just now?” when it is uncertain. More generally, the idea of explicitly asking me for clarification when it is uncertain would go a long way toward making these interactions feel more human; that’s what I (a representative human!) would do if I were on a noisy phone connection with someone and didn’t hear them clearly.</li><li><strong>Overly agreeable artificial tone</strong>: Lastly, it’s still ChatGPT under the hood, so all the regular limitations of ChatGPT apply here. Most notably, ChatGPT is tuned to be overly friendly and overly agreeable (sounding like a customer service agent) so it will simply go along with whatever you assert. Thus, by default it will not be good at pushing back on you or challenging your thinking in any meaningful ways, just like how you wouldn’t expect a customer service agent to challenge what you say. Moreover, the overly friendly tone of its responses could come off as insincere and almost sarcastic at times, even though that wasn’t the designers’ intent. Relatedly, it had a tendency to ask me superficial questions after it responds, which sound mildly condescending and break the flow of our chat, like, “Sooo, what do YOU think about the San Diego Zoo? What’s YOUR favorite part of the zoo?!?” … when a normal human wouldn’t break the conversational flow so awkwardly like that. Lastly, ChatGPT is trained on data on the public internet (and can also browse the web to get more updated web contents), so it won’t do as well if you’re asking about things that haven’t been discussed much online.</li></ul> <p>To summarize the above limitations, <em>chatting with ChatGPT on my phone felt like using a walkie-talkie over a noisy channel to talk to an overly agreeable but socially unaware customer service agent who has extensive knowledge about the contents of the public internet.</em></p> <h3><strong>Parting Thoughts: Cautiously Optimistic About the Future</strong></h3> <p>Despite these limitations, I’m excited to see what’s in store for future voice interfaces to LLM-based AI tools like ChatGPT. My early experiences of talking with ChatGPT while driving gave me a glimpse into what many of us have seen growing up in sci-fi shows such as <em>Star Trek</em>, where people can talk to an omnipresent computer to ask questions, hold conversations, or issue commands. Hands-free operation isn’t useful only while driving—it can <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://dl.acm.org/doi/10.1145/329124.329126" target="_blank">make computing truly ubiquitous</a> by letting us seamlessly interact with computation while we are in the midst of doing housework, cooking, or childcare; and it can make computing more accessible to broader groups of people, such as those with mobility impairments.</p> <p>We still have a long way to go, though. Right now the ChatGPT iPhone app isn’t hooked up to external tools beside a basic web browser, but with the <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://openai.com/blog/introducing-gpts" target="_blank">recently announced GPT store</a> (and likely upcoming LLM app stores from other companies) it will soon be possible to hook up LLMs to a variety of tools that can manage our emails, shopping lists, personal finances, home automation, and more. Recent research has started exploring these ideas by connecting ChatGPT to home assistants such as Amazon Alexa (<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2309.13879" target="_blank">2023 arXiv paper PDF</a>). Another promising line of work is better context awareness: for instance, Meta and Ray-Ban recently announced <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.meta.com/smart-glasses/" target="_blank">new Smart Glasses</a> which allow users to chat with an AI assistant that can see what they are seeing (<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.youtube.com/watch?v=pgiWqkvIclk&ab_channel=TheVerge" target="_blank">review from <em>The Verge</em></a>). In my driving scenario, you could imagine wearing these glasses and having the AI act more like a passenger sitting alongside you in the car seeing what you see rather than someone on the other end of a phone call. Critically, a passenger can pause the conversation and tell you to watch the road more carefully if they see a possible danger ahead; a future AI powered by such smart glasses may be able to do the same thing. Alternatively, cars are now starting to directly embed AI into entertainment systems (e.g., <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.volkswagen-newsroom.com/en/press-releases/world-premiere-at-ces-volkswagen-integrates-chatgpt-into-its-vehicles-18048" target="_blank">Volkswagen announcement at CES 2024</a>), so future iterations could integrate cameras and 3D tracking to complement LLMs. One could also imagine smartglasses-based multimodal interactions where you point to objects in any physical environment and start conversations with the AI assistant about your surroundings (check out this <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.youtube.com/shorts/XKGJTMJVRBs" target="_blank">MKBHD YouTube Short showing AI chat with smart glasses</a>).</p> <p>Of course, these increasingly intense levels of AI interaction and automation come with risks, such as user overreliance, unintended command execution, mental or physical health hazards, and security/privacy violations. Thus, it will be important to design ways to both manage those risks and educate users about how to safely operate these increasingly powerful systems. Thank you very much for reading. Sooo, what do YOU think about ChatGPT’s voice mode?!? What are YOUR favorite and least favorite parts?</p>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/i-actually-chatted-with-chatgpt/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Can Language Models Replace Compilers?</title> <link>https://www.oreilly.com/radar/can-language-models-replace-compilers/</link> <comments>https://www.oreilly.com/radar/can-language-models-replace-compilers/#respond</comments> <pubDate>Tue, 09 Jan 2024 13:14:10 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[AI & ML]]></category> <category><![CDATA[Artificial Intelligence]]></category> <category><![CDATA[Commentary]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15317</guid> <description><![CDATA[Kevlin Henney and I recently discussed whether automated code generation, using some future version of GitHub Copilot or the like, could ever replace higher-level languages. Specifically, could ChatGPT N (for large N) quit the game of generating code in a high-level language like Python and produce executable machine code directly, like compilers do today? It’s […]]]></description> <content:encoded><![CDATA[<p>Kevlin Henney and I recently discussed whether automated code generation, using some future version of GitHub Copilot or the like, could ever replace higher-level languages. Specifically, could ChatGPT N (for large N) quit the game of generating code in a high-level language like Python and produce executable machine code directly, like compilers do today?</p> <p>It’s not really an academic question. As coding assistants become more accurate, it seems likely to assume that they will eventually stop being “assistants” and take over the job of writing code. That will be a big change for professional programmers—though writing code is a small part of what programmers actually do. To some extent, it’s happening now: ChatGPT 4’s “Advanced Data Analysis” can generate code in Python, run it in a sandbox, collect error messages, and try to debug it. Google’s Bard has similar capabilities. Python is an interpreted language, so there’s no machine code, but there’s no reason this loop couldn’t incorporate a C or C++ compiler.</p> <p>This kind of change has happened before: in the early days of computing, programmers “wrote” programs by plugging in wires, then by toggling in binary numbers, then by writing assembly language code, and finally (in the late 1950s) using early programming languages like COBOL (1959) and FORTRAN (1957). To people who programmed using circuit diagrams and switches, these early languages looked as radical as programming with generative AI looks today. COBOL was—literally—an early attempt to make programming as simple as writing English.</p> <p>Kevlin made the point that higher-level languages are a “repository of determinism” that we can’t do without—at least, not yet. While a “repository of determinism” sounds a bit evil (feel free to come up with your own name), it’s important to understand why it is needed. At almost every stage of programming history, there has been a repository of determinism. When programmers wrote in assembly language, they had to look at the binary 1s and 0s to see exactly what the computer was doing. When programmers wrote in FORTRAN (or, for that matter, C), the repository of determinism moved higher: the source code expressed what programmers wanted and it was up to the compiler to deliver the correct machine instructions. However, the status of this repository was still shaky. Early compilers were not as reliable as we’ve come to expect. They had bugs, particularly if they were optimizing your code (were optimizing compilers a forerunner of AI?). Portability was problematic at best: every vendor had its own compiler, with its own quirks and its own extensions. Assembly was still the “court of last resort” for determining why your program didn’t work. The repository of determinism was only effective for a single vendor, computer, and operating system.<sup>1</sup> The need to make higher-level languages deterministic across computing platforms drove the development of language standards and specifications.</p> <p>These days, very few people need to know assembler. You need to know assembler for a few tricky situations when writing device drivers or to work with some dark corners of the operating system kernel, and that’s about it. But while the way we program has changed, the structure of programming hasn’t. Especially with tools like ChatGPT and Bard, we still need a repository of determinism, but that repository is no longer assembly language. With C or Python, you can read a program and understand exactly what it does. If the program behaves in unexpected ways, it’s much more likely that you’ve misunderstood some corner of the language’s specification than that the C compiler or Python interpreter got it wrong. And that’s important: that’s what allows us to debug successfully. The source code tells us exactly what the computer is doing, at a reasonable layer of abstraction. If it’s not doing what we want, we can analyze the code and correct it. That may require rereading Kernighan and Ritchie, but it’s a tractable, well-understood problem. We no longer have to look at the machine language—and that’s a very good thing, because with instruction reordering, speculative execution, and long pipelines, understanding a program at the machine level is a lot more difficult than it was in the 1960s and 1970s. We need that layer of abstraction. But that abstraction layer must also be deterministic. It must be completely predictable. It must behave the same way every time you compile and run the program.</p> <p>Why do we need the abstraction layer to be deterministic? Because we need a reliable statement of exactly what the software does. All of computing, including AI, rests on the ability of computers to do something reliably and repeatedly, millions, billions, or even trillions of times. If you don’t know exactly what the software does—or if it might do something different the next time you compile it—you can’t build a business around it. You certainly can’t maintain it, extend it, or add new features if it changes whenever you touch it, nor can you debug it.</p> <p>Automated code generation doesn’t yet have the kind of reliability we expect from traditional programming; Simon Willison calls this “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://simonwillison.net/2023/Dec/31/ai-in-2023/" target="_blank">vibes-based development</a>.” We still rely on humans to test and fix the errors. More to the point: you’re likely to generate code many times en route to a solution; you’re not likely to take the results of your first prompt and jump directly into debugging any more than you’re likely to write a complex program in Python and get it right the first time. Writing prompts for any significant software system isn’t trivial; the prompts can be very lengthy, and it takes several tries to get them right. With the current models, every time you generate code, you’re likely to get something different. (Bard even gives you several alternatives to choose from.) The process isn’t repeatable. How do you understand what the program is doing if it’s a different program each time you generate and test it? How do you know whether you’re progressing towards a solution if the next version of the program may be completely different from the previous?</p> <p>It’s tempting to think that this variation is controllable by setting a variable like GPT-4’s “temperature” to 0; “temperature” controls the amount of variation (or originality, or unpredictability) between responses. But that doesn’t solve the problem. Temperature only works within limits, and one of those limits is that the prompt must remain constant. Change the prompt to help the AI generate correct or well-designed code, and you’re outside of those limits. Another limit is that the model itself can’t change—but models change all the time, and those changes aren’t under the programmer’s control. All models are eventually updated, and there’s no guarantee that the code produced will stay the same across updates to the model. An updated model is likely to produce completely different source code. That source code will need to be understood (and debugged) on its own terms.</p> <p>So the natural language prompt can’t be the repository of determinism. This doesn’t mean that AI-generated code isn’t useful; it can provide a good starting point to work from. But at some point, programmers need to be able to reproduce and reason about bugs: that’s the point at which you need repeatability and can’t tolerate surprises. Also at that point, programmers will have to refrain from regenerating the high-level code from the natural language prompt. The AI is effectively creating a first draft, and that may (or may not) save you effort compared to starting from a blank screen. Adding features to go from version 1.0 to 2.0 raises a similar problem. Even the largest context windows can’t hold an entire software system, so it’s necessary to work one source file at a time—exactly the way we work now, but again, with the source code as the repository of determinism. Furthermore, it’s difficult to tell a language model what it’s allowed to change and what should remain untouched: “modify this loop only, but not the rest of the file” may or may not work.</p> <p>This argument doesn’t apply to coding assistants like GitHub Copilot. Copilot is aptly named: it’s an assistant to the pilot, not the pilot. You can tell it precisely what you want done, and where. When you use ChatGPT or Bard to write code, you’re not the pilot or the copilot; you’re the passenger. You can tell a pilot to fly you to New York, but from then on, the pilot is in control.</p> <p>Will generative AI ever be good enough to skip the high-level languages and generate machine code? Can a prompt replace code in a high-level language? After all, we’re already seeing a tools ecosystem that has prompt repositories, no doubt with version control. It’s possible that generative AI will eventually be able to replace programming languages for day-to-day scripting (“Generate a graph from two columns of this spreadsheet”). But for larger programming projects, keep in mind that part of human language’s value is its ambiguity, and a programming language is valuable precisely because it isn’t ambiguous. As generative AI penetrates further into programming, we will undoubtedly see stylized dialects of human languages that have less ambiguous semantics; those dialects may even become standardized and documented. But “stylized dialects with less ambiguous semantics” is really just a fancy name for prompt engineering, and if you want precise control over the results, prompt engineering isn’t as simple as it seems. We still need a repository of determinism, a layer in the programming stack where there are no surprises, a layer that provides the definitive word on what the computer will do when the code executes. Generative AI isn’t up to that task. At least, not yet. </p> <hr class="wp-block-separator" /> <h3>Footnote</h3> <ol><li>If you were in the computing industry in the 1980s, you may remember the need to “reproduce the behavior of VAX/VMS FORTRAN bug for bug.”</li></ol>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/can-language-models-replace-compilers/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item> <title>Radar Trends to Watch: January 2024</title> <link>https://www.oreilly.com/radar/radar-trends-to-watch-january-2024/</link> <comments>https://www.oreilly.com/radar/radar-trends-to-watch-january-2024/#respond</comments> <pubDate>Thu, 04 Jan 2024 11:08:07 +0000</pubDate> <dc:creator><![CDATA[Mike Loukides]]></dc:creator> <category><![CDATA[Radar Trends]]></category> <category><![CDATA[Signals]]></category> <guid isPermaLink="false">https://www.oreilly.com/radar/?p=15310</guid> <description><![CDATA[More large language models. Always more large language models. Will the new year be any different? But there is a difference in this month’s AI news: there’s an emphasis on tools that make it easy for users to use models. Whether it’s just tweaking a URL so you can ask questions of a paper on […]]]></description> <content:encoded><![CDATA[<p>More large language models. Always more large language models. Will the new year be any different? But there is a difference in this month’s AI news: there’s an emphasis on tools that make it easy for users to use models. Whether it’s just tweaking a URL so you can ask questions of a paper on arXiv or using LLamafile to run a model on your laptop (make sure you have a lot of memory!) or using the Notebook Language Model to query your own documents, AI is becoming widely accessible—and not just a toy with a web interface.</p> <h2>Artificial Intelligence</h2> <ul><li>Adding talk2 to the start of any arXiv URL (e.g., talk2arxiv.org) loads the paper into an AI chat application so you can talk to it. This is a very clever <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/evanhu1/talk2arxiv" target="_blank">application of the RAG pattern</a>.</li><li>Google’s Autonomous Vehicle startup, Waymo, has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/cars/2023/12/human-drivers-crash-a-lot-more-than-waymos-software-data-shows/" target="_blank">reported</a> a total of three minor injuries to humans in over 7 million miles of driving. This is clearly not Tesla, not Uber, not Cruise.</li><li>Google’s DeepMind has used a large language model to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2023/12/14/1085318/google-deepmind-large-language-model-solve-unsolvable-math-problem-cap-set/" target="_blank">solve</a> a previously unsolved <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.nature.com/articles/s41586-023-06924-6" target="_blank">problem</a> in mathematics. This is arguably the first time a language model has created information that didn’t previously exist.</li><li>The creator of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/Mozilla-Ocho/llamafile" target="_blank">llamafile</a> has offered a set of <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://justine.lol/oneliners/" target="_blank">one-line bash scripts</a> for laptop-powered AI. </li><li>Microsoft has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/" target="_blank">released</a> a small language model named Phi-2. Phi-2 is a 2.7B parameter model that has been trained extensively on “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.microsoft.com/en-us/research/publication/textbooks-are-all-you-need/" target="_blank">textbook-quality data</a>.” Without naming names, they claim performance superior to Llama 2.</li><li>Claude, Anthropic’s large language model, <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://docs.anthropic.com/claude/docs/using-claude-for-sheets" target="_blank">can be used in Google Sheets</a> via a browser extension.</li><li>The <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://notebooklm.google.com/?pli=1" target="_blank">Notebook Language Model</a> is a RAG implementation designed for individuals. It is a Google notebook (similar to Colab or Jupyter) that allows you to upload documents and then ask questions about those documents.</li><li>The European Union is about to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2023/12/11/1084942/five-things-you-need-to-know-about-the-eus-new-ai-act/" target="_blank">pass its AI Act</a>, which will be the world’s most significant attempt to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.europarl.europa.eu/news/en/press-room/20231206IPR15699/artificial-intelligence-act-deal-on-comprehensive-rules-for-trustworthy-ai" target="_blank">regulate</a> artificial intelligence.</li><li>Mistral has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://mistral.ai/news/mixtral-of-experts/" target="_blank">released</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/docs/transformers/model_doc/mixtral" target="_blank">Mixtral</a> 8x7B, a mixture-of-experts model in which the model first determines which of eight sets of 7 billion parameters will generate the best response to a prompt. The results compare well to Llama 2. Mistral 7B and Mixtral can be run with <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/Mozilla-Ocho/llamafile" target="_blank">Llamafile</a>.</li><li>Meta has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://ai.meta.com/blog/purple-llama-open-trust-safety-generative-ai/" target="_blank">announced</a> Purple Llama, a project around trust and safety for large language models. They have released a set of benchmarks for evaluating model safety, along with a classifier for filtering unsafe input (prompts) and model output.</li><li>The <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://postgresml.org/blog/introducing-the-openai-switch-kit-move-from-closed-to-open-source-ai-in-minutes" target="_blank">Switch Kit</a> is an open source software development kit that allows you to replace OpenAI with an open source language model easily.</li><li>Google has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://deepmind.google/technologies/gemini/#build-with-gemini" target="_blank">announced</a> that its multimodal Gemini AI model is available to software developers via their AI Studio and Vertex AI.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenextweb.com/news/new-ai-tool-democratised-image-generation" target="_blank">Progressive upscaling</a> is a technique for starting with a low-resolution image and using AI to increase the resolution. It reduces the computational power needed to generate high-resolution images. It has been implemented as a plug-in to Stable Diffusion called <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arxiv.org/abs/2311.16973" target="_blank">DemoFusion</a>.</li><li>The internet enabled mass surveillance, but that still leaves you with exabytes of data to analyze. According to Bruce Schneier, AI’s ability to analyze and draw conclusions from that data enables “<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.schneier.com/blog/archives/2023/12/the-internet-enabled-mass-surveillance-ai-will-enable-mass-spying.html" target="_blank">mass spying</a>.”</li><li>A group of over 50 organizations, including Meta, IBM, and Hugging Face, has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://9to5mac.com/2023/12/05/ai-alliance/" target="_blank">formed the AI Alliance</a> to focus on the development of open source models.</li><li>DeepMind has built an AI system that demonstrates <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2023-12-deepmind-ai-social-capabilities.html" target="_blank">social learning</a>: the ability to learn how to solve a problem by observing an expert.</li><li>Are neural networks the only way to build artificial intelligence? <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://hivekit.io/blog/building-ai-without-a-neural-network/" target="_blank">Hivekit</a> is building tools for a distributed spatial rules engine that can provide the communications layer for hives, swarms, and colonies.</li><li>The proliferation of AI testing tools continues with <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2023-12-ai-gaia-benchmark-tool-general.html" target="_blank">Gaia</a>, a benchmark suite intended to determine whether AI systems are, indeed, intelligent. The benchmark consists of a set of questions that are easy for humans to answer but difficult for computers.</li><li>Meta has just published a suite of multilingual spoken language models called <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://ai.meta.com/research/seamless-communication/" target="_blank">Seamless</a>. The models are capable of near real-time translation and claim to be more faithful to natural human expression.</li><li>In an experiment simulating a stock market, a stock-trading <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.schneier.com/blog/archives/2023/12/ai-decides-to-engage-in-insider-trading.html" target="_blank">AI system engaged in “insider trading”</a> after being put under pressure to show greater returns and receiving “tips” from company “employees.”</li><li>What’s the best way to run a large language model on your laptop? Simon Willison <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://simonwillison.net/2023/Nov/29/llamafile/" target="_blank">recommends</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://huggingface.co/jartine/llava-v1.5-7B-GGUF/blob/main/llamafile-server-0.1-llava-v1.5-7b-q4" target="_blank">llamafile</a>, which packages a model together with the weights as a single (large) executable that works on multiple operating systems.</li><li>Further work on <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html" target="_blank">extracting training data from ChatGPT</a>, this time against the production model, shows that these systems may be opaque, but they aren’t quite “black boxes.”</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://press.aboutamazon.com/2023/11/aws-announces-amazon-q-to-reimagine-the-future-of-work" target="_blank">Amazon Q</a> is a new large language model that includes a chatbot and other tools to aid office workers. It can be customized by individual businesses that subscribe to the service so that it has access to their proprietary data.</li></ul> <h2>Programming</h2> <ul><li>A new language superset: <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://pluto-lang.org/docs/Introduction" target="_blank">Pluto</a> is a superset of Lua. Supersetting may be the “new thing” in language design: TypeScript, Mojo, and a few others (including the first versions of C++) come to mind.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/the-new-age-of-virtualization/" target="_blank">Virtualization within containers orchestrated by Kubernetes</a>: Can you imagine a Kubernetes cluster running within a Docker container? Is that a good thing or evidence of how a stack’s complexity can grow without bounds?</li><li>Google engineers <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://dl.acm.org/doi/10.1145/3593856.3595909?utm_source=thenewstack&utm_medium=website&utm_content=inline-mention&utm_campaign=platform" target="_blank">propose</a> an <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/year-in-review-was-2023-a-turning-point-for-microservices/" target="_blank">alternative to microservices</a>: limited monoliths that are deployed by an automated runtime that determines where and when to instantiate them. As Kelsey Hightower said, deployment architecture becomes an implementation detail.</li><li>The OpenBao project is intended to be an open source fork of HashiCorp’s Vault, analogous to the OpenTofu fork of Terraform. There is <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/meet-openbao-an-open-source-fork-of-hashicorp-vagrant/" target="_blank">speculation</a> that IBM will back both projects.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.biscuitsec.org/" target="_blank">Biscuit authorization</a> is a distributed authorization protocol that is relatively small, flexible, and is designed for use in distributed systems. Any node can validate a Biscuit token using only public information.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://gokrazy.org/" target="_blank">gokrazy</a> is a minimal Go runtime environment for the Raspberry Pi and (some) PCs. It minimizes maintenance by eliminating everything that isn’t needed to compile and run Go programs.</li><li>You very clearly don’t need this: A <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/nst/bfps" target="_blank">Brainfuck interpreter written in PostScript</a>. (If you really must know, Brainfuck is arguably the world’s most uncomfortable programming language, and PostScript is the language your computer sends to a printer.)</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://baserow.io/" target="_blank">Baserow</a> is a no-code, open source tool that combines a spreadsheet with a database. It’s similar to Airtable.</li><li>New programming language of the month: <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://wasmer.io/posts/onyxlang-powered-by-wasmer" target="_blank">Onyx</a> is a new programming language designed to generate WebAssembly (Wasm), using <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://wasmer.io/" target="_blank">Wasmer</a> as the underlying runtime.</li></ul> <h2>Web</h2> <ul><li>Anil Dash predicts that <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.rollingstone.com/culture/culture-commentary/internet-future-about-to-get-weird-1234938403/" target="_blank">the internet is about to get weird again</a>—the way it should be. Power is shifting from the entrenched, heavily funded “walled gardens” and back to people who just want to be creative.</li><li>Meta’s Threads has begun to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theverge.com/2023/12/13/24000120/threads-meta-activitypub-test-mastodon" target="_blank">test integration with ActivityPub</a>, which will make it accessible to Mastodon servers. </li><li>The <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2023/12/21/1084525/internet-whimsy-html-energy/" target="_blank">HTML Energy</a> movement attempts to reclaim the creativity of the early web by building sites from scratch with HTML and abandoning high-powered web frameworks.</li><li>The best WebAssembly runtime might be <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://00f.net/2023/12/11/webassembly-compilation-to-c/" target="_blank">no runtime</a> at all: just transpile it to C. </li></ul> <h2>Security</h2> <ul><li>Researchers have <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/security/2023/12/hackers-can-break-ssh-channel-integrity-using-novel-data-corruption-attack/" target="_blank">discovered</a> a man-in-the-middle attack against SSH, one of the foundations of cybersecurity.</li><li>A new version of SSH (<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/francoismichel/ssh3" target="_blank">SSH3</a>) promises to be faster and more feature-rich. It is based on HTTP/3 and written in Go.</li><li>Security researchers have <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2023-12-customized-gpt-vulnerability.html" target="_blank">demonstrated</a> two important vulnerabilities in OpenAI’s custom GPTs. Malicious actors can extract system prompts, and they can force it to leak uploaded files and other data.</li><li>Meta has made end-to-end encryption (E2EE) the default for all users of Messenger and Facebook messaging. Their E2EE implementation is based on Signal’s. They have built a new <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.bleepingcomputer.com/news/security/meta-rolls-out-default-end-to-end-encryption-on-messenger-facebook/" target="_blank">storage and retrieval service</a> for encrypted messages.</li><li>A chatbot driven by a jailbroken language model can be used to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.scientificamerican.com/article/jailbroken-ai-chatbots-can-jailbreak-other-chatbots/" target="_blank">jailbreak other chatbots</a>. Language models are very good at coming up with prompts that get other models to go outside their boundaries, with success rates of 40% to 60%. AI security will be a key topic this year.</li></ul> <h2>Quantum Computing</h2> <ul><li>IBM has developed a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/science/2023/12/ibm-adds-error-correction-to-updated-quantum-computing-roadmap/" target="_blank">1121 qubit quantum processor</a>, along with a system built from three 133 qubit processor chips that greatly improves the accuracy of quantum gates. Working quantum computers will probably require over a million qubits, but this is a big step forward.</li><li>A research group has <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://arstechnica.com/science/2023/12/quantum-computer-performs-error-resistant-operations-with-logical-qubits/" target="_blank">announced</a> that it can perform computations on 48 logical (i.e., error-corrected) qubits. While there are a number of limitations to their work, it’s an important step toward practical quantum computing.</li><li>Two posts about post-<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blog.cryptographyengineering.com/2023/10/06/to-schnorr-and-beyond-part-1/" target="_blank">quantum</a> <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://blog.cryptographyengineering.com/2023/11/30/to-schnorr-and-beyond-part-2/" target="_blank">cryptography</a> explain what it’s about.</li></ul> <h2>Brains</h2> <ul><li>Researchers have developed a noninvasive system that can <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://techxplore.com/news/2023-12-portable-non-invasive-mind-reading-ai-thoughts.html" target="_blank">turn human thought into text</a>. Users wear a cap with sensors that generates EEG data. Accuracy isn’t very high yet, but it is already superior to other thought-to-speech technologies.</li><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.technologyreview.com/2023/12/11/1084926/human-brain-cells-chip-organoid-speech-recognition/" target="_blank">Artificial neural networks with brains</a>: Researchers connected cultured human brain cells (organoids) to an interface that allowed them to give the organoids audio data. They found that it was able to recognize vowel sounds.</li></ul> <h2>Virtual and Augmented Reality</h2> <ul><li><a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://github.com/PixarAnimationStudios/OpenUSD" target="_blank">OpenUSD</a> is an open source <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://thenewstack.io/openusd-could-enable-a-real-metaverse/" target="_blank">standard for scene representation</a> that could enable a real metaverse, not the proprietary walled garden imagined by last year’s metaverse advocates.</li></ul>]]></content:encoded> <wfw:commentRss>https://www.oreilly.com/radar/radar-trends-to-watch-january-2024/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> </channel></rss> If you would like to create a banner that links to this page (i.e. this validation result), do the following:
Download the "valid RSS" banner.
Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)
Add this HTML to your page (change the image src attribute if necessary):
If you would like to create a text link instead, here is the URL you can use:
http://www.feedvalidator.org/check.cgi?url=http%3A//feeds.feedburner.com/oreilly/radar/atom
