AWS IoT Solutions

IoT offers unparalleled opportunities for every industry to address its business challenges. With the explosion of devices, there is a need for a solution to connect, collect, store, and analyze device data. Amazon Web Services provides various services to help connected devices interact easily and securely with cloud applications and other devices to meet user scenarios.

That being said, every solution architect in this field knows the capabilities and reliability of the AWS cloud. Migrating or designing IoT solutions on the AWS platform allows people to focus on their core business without infrastructure management and monitoring, ensuring high customer availability. Regardless of the solution being designed, the best platform should be chosen to maintain the stability of the solution. AWS is such a platform.

When designing IoT solutions using AWS, there are a few practices to consider. If the right AWS services are used to meet customer needs, IoT solutions can deliver results in a more secure, reliable, and scalable manner.

Design for reliable large-scale operation

IoT systems must handle high-speed and massive amounts of data devices and gateways capture. Overflow of incoming data can be expected due to sudden business growth or sometimes due to malicious attacks. In this case, the cloud system architecture should be scalable to handle such data.

The best approach is to send data to queues, buffers, or real-time in-memory databases before storing it in storage. This helps implement real-time events and slows the data insertion rate to prevent database crashes or slow response times.

Devices can publish data to AWS Kinesis, or AWS IoT rules can forward data to AWS SQS and Kinesis to store it in AWS S3, Redshift, DataLake, or Elasticsearch for time-series storage. These data stores can generate custom dashboards or AWS QuickSight dashboards.

Route large data volumes through data pipelines

Using incoming data from device topics directly to a single service prevents the system from achieving full scalability. Sometimes, this approach limits the system's availability during failures and data flooding events.

The AWS IoT Rules Engine is designed to connect endpoints to AWS IoT Core in a scalable way. However, all AWS services have different data flow properties and their respective pros and cons. Not all services can be used as a single entry point for the system. Sometimes, it can lead to subsequent unrecoverable failures. For example, for large volumes of data, consider buffering (ElastiCache) or queuing (SQS) incoming data before calling other services so that you can recover from subsequent failures.

The AWS IoT Rules Engine allows parallel triggering of multiple AWS services such as Lambda, S3, Kinesis, SQS, and SNS. Once the IoT system captures data, it enables AWS endpoints (other AWS services) to process and transform the data. This allows data to be stored in multiple data stores simultaneously.

The safest and best way to ensure all data is processed and stored is to redirect all device topic data to SNS, designed to handle data flooding, ensuring reliable maintenance, processing, and delivery of incoming data to the appropriate channels. Multiple SNS topics, SQS queues, and Lambda for different/a group of AWS device topics can be used to make it more scalable. Before processing, data should be stored in secure storage, such as Queue, Amazon Kinesis, Amazon S3, and Amazon Redshift. This approach ensures that data is not lost due to message flooding, unnecessary exception codes, or deployment issues.

Automate device configuration and upgrades

As the business grows and numerous devices connect to the IoT ecosystem, manual processes for device configuration, bootstrapping software, security configuration, rules operation settings, and device OTA upgrades become unfeasible. Minimizing human interaction during initialization and upgrade processes saves time and cost.

Designing built-in features in devices for automatic provisioning and leveraging appropriate tools provided by AWS to handle device provisioning and management allows the system to achieve the required operational efficiency with minimal human intervention.

AWS IoT provides a set of features that can be used for bulk import, including policies that can be integrated with dashboards or manufacturing processes, where devices can be pre-registered to AWS IoT, and certificates can be installed in devices. Later, the device provisioning flow can declare the device and attach it to a user or any other entity. AWS provides tools to trigger and track device OTA upgrades.

Adopt a scalable architecture for custom components

When an IoT system connects to external world devices, the scope doesn't end with connecting, controlling, and reporting devices. Consider adopting the latest technologies like data science and machine learning or integrating third-party components in IoT systems like IFTTT, Alexa, or Google Home. The IoT architecture should ensure that external components can be easily integrated into the solution without performance bottlenecks.

Check for offline access and processing.

It's only sometimes necessary to process all machine data in the cloud. In many cases, a continuous internet connection may not be available. For such scenarios, add AWS Greengrass at the edge. Greengrass processes and filters data locally at the edge, reducing the need to send all device data upstream. You can capture all data, retain it for a limited time, and send it to the cloud in case of error events or on-demand/upon request. If time-series data is required, you can schedule a periodic process to send device data to the cloud, which can be used for future enhanced features like AWS machine learning models and cloud analytics tools.

Choose the right data storage.

IoT systems generate high-speed, large-capacity, and diverse data. Each IoT device or device topic may have different formats, which may not be manageable through a single database or similar type of data storage. Architects should be cautious when choosing database formats and data storage. Sometimes, single data storage works well, or a mix of data storage for different purposes helps achieve high throughput. Frequently used static data can be stored in ElastiCache, which helps improve performance. This approach contributes to the system's scalability and maintainability.

Filter and transform data before processing.

All incoming data to the IoT system may need processing or transformation, after which it can be redirected to storage as is. AWS IoT rules provide actions to redirect messages to different AWS services. Architects should lurk all data in different forms, i.e., data that needs processing, ignored/static data (like Config), and direct storage.

AWS IoT service architecture

AWS IoT helps achieve fast device connections, secure data ingestion, easy device management, multi-protocol support, and more.