Source: Ar_TH/stock.adobe.com
The history of computing can be told through many lenses. Take, for instance, the cat-and-mouse game between hardware and software. New hardware is introduced, and it takes a while for software to take full advantage of its computing horsepower. Then, the software becomes resource-hungry and must wait for the hardware to catch up. Another example is the back-and-forth between local and remote computational capability. While the terminology may change—cloud versus remote and local versus the edge—the general concepts remain relevant. Essentially, edge computing is about maximizing data processing as physically close to the source as possible.
Edge computing is an iteration of computational evolution where data are not just mouse clicks and keystrokes, but rather sensor telemetry, camera images, and video streams, among many other sources. So, in an era of nearly ubiquitous wireless internet access in many areas of the globe, why not just package all the data and send it to the cloud? After all, we have been doing that since 2006, after the emergence of reliable cloud infrastructure, such as Amazon Web Services (AWS), Google Coud, and Microsoft Azure. Answering this question, however, is not so simple.
To answer that question, let’s look at the rise of the internet itself. In the 1990s, there was a latency issue as websites started springing up on the World Wide Web. While a website hosted on a server in the United States might be accessible from Japan, downloading the site to the browser could take a relatively long time. This delay, known as latency, could be exacerbated if your network is very active with other users. Then, if the site became popular, the additional traffic would only make things worse. To reduce this latency and improve the overall experience, locally available content distribution networks (CDNs) were devised to cache copies of websites (or other content like movies or documents) closer to the end users accessing them.
With that foundation, let’s examine what makes edge computing different from what has come before. Around the time smartphones and associated peripherals—like wearables, Internet of Things (IoT) cameras, smart lightbulbs—became common, a fundamental shift occurred in the source and directional flow of data. Instead of accessing static content from a remote server to view locally in a web browser, these “smart devices” started generating lots of data from their multiple onboard sensors.
The problem was that these devices were not actually so smart. Sensors may generate data, but data is not information or knowledge. Data must be processed for it to be useful, and, for smart devices, that processing occurs in the cloud, especially with the advent of computationally intensive artificial intelligence (AI) and machine learning (ML) algorithms. Suddenly, lots of devices with lots of data were pushing lots of bits up to the cloud.
All this brings us back to that pesky little problem of latency, specifically with inferencing. In computing, inference is when a trained AI model is presented with new inputs to produce probabilistic outputs. It is the most end-user-focused aspect of AI/ML technologies and has historically been computationally intensive. Therefore, getting inferencing right is crucial. Initially, the only viable option was to send raw data to the cloud to leverage the computational horsepower of server-grade hardware and then send the outputs back to the local device. But embedded hardware has since become more powerful, and ML algorithms have developed greater efficiency. This convergence has created several noteworthy benefits that make edge computing a viable alternative to sending all that data to the cloud for processing.
The emergence of new hardware and software over the past few years has enabled the increased use of edge computing, making it possible to run complex algorithms like neural networks on relatively inexpensive, battery-operated devices. The following are some of the current edge-native hardware:
Still, hardware is nothing without software. AI workloads at the edge require optimized software frameworks for low-power inferencing, with the following popular frameworks available:
However, for crucial operations that can’t wait for the cloud, such as real-time processing for AI, video analytics, and robotics, processing at the edge doesn’t mean that data is never sent to the cloud. There are still good reasons for sending data to the cloud via edge gateways:
Edge-oriented communication protocols and message brokers, including the following, ensure this communication is done efficiently:
To ensure security, performance, and flexibility at the edge, specialized operating systems and runtime environments for edge deployments have emerged. These environments are purpose-built to run reliably on resource-constrained devices, support containerized workloads, and enable secure software delivery and lifecycle management in the field. Some of the widely used platforms include:
Lastly, to manage the growing complexity of distributed edge computing systems, major cloud providers and specialized platforms now offer orchestrated frameworks that bridge the edge and cloud. These platforms support device provisioning, workload deployment, secure communication, and lifecycle management.
Edge computing represents a pivotal shift in how we design, build, and deploy intelligent systems—and for engineers, it opens an exciting new frontier. With the ability to process data locally, in real time, and on resource-constrained devices, edge computing directly brings advanced capabilities like AI, automation, and analytics to the environments where data is generated. Beyond its technological significance, edge computing is also important from legal and business perspectives, with the rise of sovereignty and privacy laws, such as the General Data Protection Regulation (GDPR).
This evolution is driven by increasingly powerful microcontrollers, energy-efficient AI accelerators, and lightweight ML frameworks. Platforms including AWS Greengrass and Azure IoT Edge, along with tools like TensorFlow Lite and NVIDIA Jetson, are democratizing access to technologies once reserved for enterprise data centers. Today, engineers can prototype and deploy sophisticated, intelligent edge applications, from autonomous robots and smart environmental monitors to connected wearables and industrial automation systems.
Ultimately, edge computing allows engineers to break free from cloud dependency, enabling more responsive, resilient, and privacy-conscious designs. It empowers creators to build systems that think locally and act instantly—an approach that is not only technically powerful but also increasingly necessary in a connected, data-driven world. As the edge continues to evolve, it offers an unprecedented opportunity for engineers to shape the next generation of intelligent, decentralized technology.
Michael Parks, P.E. is the co-founder of Green Shoe Garage, a custom electronics design studio and embedded security research firm located in Western Maryland. He produces the Gears of Resistance Podcast to help raise public awareness of technical and scientific matters. Michael is also a licensed Professional Engineer in the state of Maryland and holds a Master’s degree in systems engineering from Johns Hopkins University.