Pankesh Patel

Bharath Sudharsan, Dineshkumar Sundaram, Pankesh Patel et al.

The majority of IoT devices like smartwatches, smart plugs, HVAC controllers, etc., are powered by hardware with a constrained specification (low memory, clock speed and processor) which is insufficient to accommodate and execute large, high-quality models. On such resource-constrained devices, manufacturers still manage to provide attractive functionalities (to boost sales) by following the traditional approach of programming IoT devices/products to collect and transmit data (image, audio, sensor readings, etc.) to their cloud-based ML analytics platforms. For decades, this online approach has been facing issues such as compromised data streams, non-real-time analytics due to latency, bandwidth constraints, costly subscriptions, recent privacy issues raised by users and the GDPR guidelines, etc. In this paper, to enable ultra-fast and accurate AI-based offline analytics on resource-constrained IoT devices, we present an end-to-end multi-component model optimization sequence and open-source its implementation. Researchers and developers can use our optimization sequence to optimize high memory, computation demanding models in multiple aspects in order to produce small size, low latency, low-power consuming models that can comfortably fit and execute on resource-constrained hardware. The experimental results show that our optimization components can produce models that are; (i) 12.06 x times compressed; (ii) 0.13% to 0.27% more accurate; (iii) Orders of magnitude faster unit inference at 0.06 ms. Our optimization sequence is generic and can be applied to any state-of-the-art models trained for anomaly detection, predictive maintenance, robotics, voice recognition, and machine vision.

Parthkumar Patel, Jayraj Dave, Shreedhar Dalal et al.

The idea of IoT world has grown to multiple dimensions enclosing different technologies and standards which can provide solutions and goal oriented intelligence to the widespread things via network or internet. In spite of different advancement in technology, challenges related to assessment of IoT solutions under real scenarios and empirical deployments still hinder their evolvement and significant expansion. To design a system that can adequately bolster substantial range of applications and be compliant with superfluity of divergent requirements and also integrating heterogeneous technologies is a difficult task. Thus, simulations and testing to design robust applications becomes paramount elements of a development process. For this, there rises a need of a tool or a methodology to test and manage the applications. This paper presents a novel approach by proposing a testbed for experimenting Internet of Things (IoT) applications. An idea of an open source test bed helps in developing an exploited and sustainable smart system. In order to validate the idea of such testbed we have also implemented two use cases.

Devam Dave, Het Naik, Smiti Singhal et al.

The objective of an online Mart is to match buyers and sellers, to weigh animals and to oversee their sale. A reliable pricing method can be developed by ML models that can read through historical sales data. However, when AI models suggest or recommend a price, that in itself does not reveal too much (i.e., it acts like a black box) about the qualities and the abilities of an animal. An interested buyer would like to know more about the salient features of an animal before making the right choice based on his requirements. A model capable of explaining the different factors that impact the price point is essential for the needs of the market. It can also inspire confidence in buyers and sellers about the price point offered. To achieve these objectives, we have been working with the team at MartEye, a startup based in Portershed in Galway City, Ireland. Through this paper, we report our work-in-progress research towards building a smart video analytic platform, leveraging Explainable AI techniques.

Devam Dave, Het Naik, Smiti Singhal et al.

With the increasing availability of structured and unstructured data and the swift progress of analytical techniques, Artificial Intelligence (AI) is bringing a revolution to the healthcare industry. With the increasingly indispensable role of AI in healthcare, there are growing concerns over the lack of transparency and explainability in addition to potential bias encountered by predictions of the model. This is where Explainable Artificial Intelligence (XAI) comes into the picture. XAI increases the trust placed in an AI system by medical practitioners as well as AI researchers, and thus, eventually, leads to an increasingly widespread deployment of AI in healthcare. In this paper, we present different interpretability techniques. The aim is to enlighten practitioners on the understandability and interpretability of explainable AI systems using a variety of techniques available which can be very advantageous in the health-care domain. Medical diagnosis model is responsible for human life and we need to be confident enough to treat a patient as instructed by a black-box model. Our paper contains examples based on the heart disease dataset and elucidates on how the explainability techniques should be preferred to create trustworthiness while using AI systems in healthcare.

Vatsal Patel, Sarth Kanani, Tapan Pathak et al.

Smart doorbells have been playing an important role in protecting our modern homes. Existing approaches of sending video streams to a centralized server (or Cloud) for video analytics have been facing many challenges such as latency, bandwidth cost and more importantly users' privacy concerns. To address these challenges, this paper showcases the ability of an intelligent smart doorbell based on Federated Deep Learning, which can deploy and manage video analytics applications such as a smart doorbell across Edge and Cloud resources. This platform can scale, work with multiple devices, seamlessly manage online orchestration of the application components. The proposed framework is implemented using state-of-the-art technology. We implement the Federated Server using the Flask framework, containerized using Nginx and Gunicorn, which is deployed on AWS EC2 and AWS Serverless architecture.

Bhavin Joshi, Tapan Pathak, Vatsal Patel et al.

The design of products and services such as a Smart doorbell, demonstrating video analytics software/algorithm functionality, is expected to address a new kind of requirements such as designing a scalable solution while considering the trade-off between cost and accuracy; a flexible architecture to deploy new AI-based models or update existing models, as user requirements evolve; as well as seamlessly integrating different kinds of user interfaces and devices. To address these challenges, we propose a smart doorbell that orchestrates video analytics across Edge and Cloud resources. The proposal uses AWS as a base platform for implementation and leverages Commercially Available Off-The-Shelf(COTS) affordable devices such as Raspberry Pi in the form of an Edge device.

Amelie Gyrard, Martin Serrano, Pankesh Patel

The Web of Things (WoT) is rapidly growing in popularity getting the interest of not only technologist and scientific communities but industrial, system integrators and solution providers. The key aspect of the WoT to succeed is the relatively, easy-to-build ecosystems nature inherited from the web and the capacity for building end-to-end solutions. At the WoT connecting physical devices such as sensors, RFID tags or any devices that can send data through the Internet using the Web is almost automatic. The WoT shared data can be used to build smarter solutions that offer business services in the form of IoT applications. In this chapter, we review the main WoT challenges, with particular interest on highlighting those that rely on combining heterogeneous IoT data for the design of smarter services and applications and that benefit from data interoperability. Semantic web technologies help for overcoming with such challenges by addressing, among other ones the following objectives: 1) semantically annotating and unifying heterogeneous data, 2) enriching semantic WoT datasets with external knowledge graphs, and 3) providing an analysis of data by means of reasoning mechanisms to infer meaningful information. To overcome the challenge of building interoperable semantics-based IoT applications, the Machine-to-Machine Measurement (M3) semantic engine has been designed to semantically annotate WoT data, build the logic of smarter services and deduce meaningful knowledge by linking it to the external knowledge graphs available on the web. M3 assists application and business developers in designing interoperable Semantic Web of Things applications. Contributions in the context of European semantic-based WoT projects are discussed and a particular use case within FIESTA-IoT project is presented.

Saurabh Chauhan, Pankesh Patel

Application development in the Internet of Things (IoT) is challenging because it involves dealing with issues that attribute to different life-cycle phases. First, the application logic has to be analyzed and then separated into a set of distributed tasks for an underlying network. Then, the tasks have to be implemented for the specific hardware. Moreover, we take different IoT applications and present development of these applications using IoTSuite. In this paper, we introduce a design and implementation of ToolSuite, a suite of tools, for reducing burden of each stage of IoT application development process. We take different class of IoT applications, largely found in the IoT literature, and demonstrate these IoT application development using IoTSuite. These applications have been tested on several IoT technologies such as Android, Raspberry PI, Arduino, and JavaSE-enabled devices, Messaging protocols such as MQTT, CoAP, WebSocket, Server technologies such as Node.js, Relational database such as MySQL, and Microsoft Azure Cloud services.

Amelie Gyrard, Pankesh Patel, Amit Sheth et al.

The Internet of Things (IoT) is experiencing fast adoption in the society, from industrial to home applications. The number of deployed sensors and connected devices to the Internet is changing our perspective and the way we understand the world. The development and generation of IoT applications is just starting and they will modify our physical and virtual lives, from how we control remotely appliances at home to how we deal with insurance companies in order to start insurance schemes via smart cards. This massive deployment of IoT devices represents a tremendous economic impact and at the same time offers multiple opportunities. However, the potential of IoT is underexploited and day by day this gap between devices and useful applications is getting bigger. Additionally, the physical and cyber worlds are largely disconnected, requiring a lot of manual efforts to integrate, find, and use information in a meaningful way. To build a connection between the physical and the virtual, we need a knowledge framework that allow bilateral understandings, devices producing data, information systems managing the data and applications transforming information into meaningful knowledge. The first column in this series in the previous issue of this magazine titled "Internet of Things to Smart IoT Through Semantic, Cognitive, and Perceptual Computing," reviews IoT growth and potential that have energized research and technology development, centered on aspects of Artificial Intelligence to build future intelligent system. This column steps back and demonstrates the benefits of using semantic web technologies to get meaningful knowledge from sensor data to design smart systems.

Pankesh Patel, Tie Luo, Umesh Bellur

A critical challenge is to enable IoT application development with minimal effort from various stakeholders involved in the development process. Several approaches to tacking this challenge have been proposed in the fields of wireless sensor networks and ubiquitous and pervasive computing, regarded as precursors to the modern day of IoT. However, although existing approaches provide a wide range of features, stakeholders have specific application development requirements and choosing an appropriate approach requires thorough evaluations on different aspects. To date, this aspect has been investigated to a limited extend. In view of this, this paper provides an extensive set of evaluations based on our previous work on IoT application development framework. Specifically, we evaluate our approach in terms of (1) development effort: the effort required to create a new application, (2) reusability: the extend to which software artifacts can be reused during application development, (3) expressiveness: the characteristics of IoT applications that can be modeled using our approach, (4) memory metrics: the amount of memory and storage a device needs to consume in order to run an application under our framework, and (5) comparison of our approach with state of the art in IoT application development on various dimensions, which does not only provide a comprehensive view of state of the art, but also guides developers in selecting an approach given application requirements in hand. We believe that the above different aspects provide the research community with insight into evaluating, selecting, and developing useful IoT frameworks and applications.

Pankesh Patel, Damien Cassou

Application development in the Internet of Things (IoT) is challenging because it involves dealing with a wide range of related issues such as lack of separation of concerns, and lack of high-level of abstractions to address both the large scale and heterogeneity. Moreover, stakeholders involved in the application development have to address issues that can be attributed to different life-cycles phases. when developing applications. First, the application logic has to be analyzed and then separated into a set of distributed tasks for an underlying network. Then, the tasks have to be implemented for the specific hardware. Apart from handling these issues, they have to deal with other aspects of life-cycle such as changes in application requirements and deployed devices. Several approaches have been proposed in the closely related fields of wireless sensor network, ubiquitous and pervasive computing, and software engineering in general to address the above challenges. However, existing approaches only cover limited subsets of the above mentioned challenges when applied to the IoT. This paper proposes an integrated approach for addressing the above mentioned challenges. The main contributions of this paper are: (1) a development methodology that separates IoT application development into different concerns and provides a conceptual framework to develop an application, (2) a development framework that implements the development methodology to support actions of stakeholders. The development framework provides a set of modeling languages to specify each development concern and abstracts the scale and heterogeneity related complexity. It integrates code generation, task-mapping, and linking techniques to provide automation. Code generation supports the application development phase by producing a programming framework that allows stakeholders to focus on the application logic, while our mapping and linking techniques together support the deployment phase by producing device-specific code to result in a distributed system collaboratively hosted by individual devices. Our evaluation based on two realistic scenarios shows that the use of our approach improves the productivity of stakeholders involved in the application development.