IEEE COMMUNICATIONS MAGAZINE, VOL. X, NO. X, XXXXX 201X 1

Enabling Cognitive Smart Cities Using Big Data

and Machine Learning: Approaches and Challenges

Mehdi Mohammadi, Graduate Student Member, IEEE, Ala Al-Fuqaha, Senior Member, IEEE

Abstract—The development of smart cities and their fast-paced

deployment is resulting in the generation of large quantities of

data at unprecedented rates. Unfortunately, most of the generated

data is wasted without extracting potentially useful information

and knowledge because of the lack of established mechanisms

by smart cities from a machine learning perspective. Especially,

we present the phenomenon of wasting unlabeled data. We argue

that semi-supervision is a must for smart city to address this

challenge. We also propose a three-level learning framework for

smart cities that matches the hierarchical nature of big data

generated by smart cities with a goal of providing different

levels of knowledge abstractions. The proposed framework is

scalable to meet the needs of smart city services. Fundamentally,

the framework benefits from semi-supervised deep reinforcement

learning where a small amount of data that has users’ feedback

smart city services and improve their performance by providing

several use cases spanning the different domains of smart cities.

We also highlight several challenges as well as promising future

research directions for incorporating machine learning and high-

level intelligence into smart city services.

I. INTRODUCTION

Smart cities provide services that benefit from the city-scale

deployment of sensors, actuators and smart objects [1]. Such

services are mainly driven by data and can be broadly classi-

fied as producers of data, consumers of data, or a combination

communications to alert nearby vehicles and pedestrians of

their presence and potential traffic hazards. In a city-scale

deployment of smart services, data is generated at high rates

which presents new challenges for smart city designers and

developers. Beyond the challenges for data management of big

Mehdi Mohammadi and Ala Al-Fuqaha are with the Department of Com-

puter Science, Western Michigan University, Kalamazoo, MI 49008 USA.

E-mail: {mehdi.mohammadi,ala.al-fuqaha}@wmich.edu.

Fig. 1. Challenges of Smart Cities from a Machine Learning perspective.

data represented by 3V’s (volume, variety, and velocity), there

are other challenges from analytics and Machine Learning

(ML) perspectives (cf. Figure 1). Unfortunately, only a small

Analogous to the waste recycling processes and standards

in urban cities, there is a need for processes and mechanisms

for data recycling in smart cities where hundreds or thousands

of Gigabytes of data is generated per second. Data analytic

methods and machine learning algorithms should be able to

extract knowledge and useful information from data to reduce

the amount of digital waste.

Despite the recent advancements in computing and storage

technologies, most of the data analytic approaches exploit

sampling methods that are efficient in terms of time complexity

used for learning and analytics in a short-term, it is unlikely

that it would be used later. It is estimated that by 2012 only

about 0.5% of all 2.8 Zettabytes (ZB) of stored data have

been analyzed and 3% of them are labeled based on a study

IEEE COMMUNICATIONS MAGAZINE, VOL. X, NO. X, XXXXX 201X 2

by IDC

1

. This highlights the challenge of potentially wasting

hidden information in 99.5% of the generated data.

Lately, there have been active discussions on the gover-

nance, management, and storage of smart cities big data,

but there is no clear answer on how to use the enormous

to predict anomalies in a city’s infrastructure considering all

collected data from the various sensor sources would help

to realize such services. Another example includes services

that predict criminal or discriminatory activities through social

media comments (e.g., Tweeter, Facebook, etc.). For such

services, considering a wider range of data is necessary, since

the criminal or discriminatory comments may constitute a

small portion of the whole data.

Smart city ecosystems have the following characteristics

from a machine learning perspective:

• They need a general, dynamic, and continuous learning

mechanism as the context of a smart city application is

not always fixed and the operating environment of smart

city applications evolves over the time.

• The data generated by smart city applications is noisy or

has some degree of uncertainty.

Based on these characteristics, we believe an integration

of DNNs, Reinforcement Learning (RL), and semi-supervised

learning can address these issues and deliver complete adaptive

solutions.

systems and systems that need to perform autonomic actions.

In reinforcement learning, there is no output (i.e., classifi-

cation) for the training data - which is the case for many

smart city applications- instead, choosing the right actions is

rewarded. The goal of a RL system is to find an action for

each state of the system such that the total reward of the

learning agent is maximized in the long-term. On the other

hand, it is infeasible or extremely tedious for the users to

provide a reward feedback for all the training data. This issue

can be addressed through the use of semi-supervised learning

https://www.emc.com/collateral/analyst-reports/idc-the-digital-universe-in-

2020.pdf

ing method to address the scarcity of annotated data in big

IEEE COMMUNICATIONS MAGAZINE, VOL. X, NO. X, XXXXX 201X 3

the communications bandwidth by not having to transmit all

raw data to the cloud, and performing real-time analytics

due to the closeness of the fog to the source of data. They

used Hidden Markov Model (HMM) approach in their model

within the hierarchy of smart city infrastructure where each

component of the smart city system is controlled by an

intelligent software agent which is deployed in the fog or

the cloud depending on the characteristics of the required

analytics (e.g., time-sensitiveness). Consequently, raw data can

be transferred to the fog or to the cloud. The running analytics

agent then returns an appropriate action to the infrastructure

devices based on predictions (e.g., adjust traffic light timing

based on traffic congestion data from the corresponding roads).

The motivation behind this architecture is that deeper levels

of data abstraction and knowledge representation can be ob-

IEEE COMMUNICATIONS MAGAZINE, VOL. X, NO. X, XXXXX 201X 4

Google Now X User-centric Various Traffic, Transit

HPE Haven OnDemand X General Various Sentiment Analysis

Research

Cognitive management

framework [3]

X X Smart city

Pattern Recognition,

Semantic Reasoning

Smart health,

Public safety,

Smart transportation

Cognitive IoT [4] X Smart city Multiagent Learning

Intelligent gateway [7] X IoT Rule-based Smart healthcare

Current work X X Smart city

Semi-supervised Deep

tained as the data travels through the smart city infrastructure.

At the highest level, a city-wide abstraction is needed to

manage the city’s resources and services on a long-term basis.

On the other hand, at the lowest level, sensors and smart

resource-constrained devices percept the environment. The

resource limitation of these devices inhibits the deployment

of complex and large learning models. Instead, several shal-

low machine learning approaches, including unsupervised and

semi-supervised methods (e.g., K-Nearest Neighbors, Support

Vector Machines, etc.) can be applied in the context of these

devices to make them smart. However, to bring analytics and

intelligence closer to the source of data (e.g., end users, IoT

resource-constrained devices) there is a need to utilize modern

and advanced learning models like deep learning. A nascent

neural networks, it is possible to integrate deep reinforcement

learning with these devices.

supervised DRL approach is also applicable at this level. Also

at this level, light-weight intelligence needs to be brought to

the IoT gateways and proxies to enable the efficient realization

of horizontal integration of services in support of smart city

applications [7].

At the cloud computing level, more complex and large-scale

machine learning and data mining frameworks and algorithms

can be integrated with semantic learning and ontologies to

extract high-level insights and patterns from the collected

are able to provide deeper abstractions of the data. Recent

use of Rectified Linear Units (ReLUs), and the introduction

of Long Short-Term Memory (LSTM) neural networks and

their variants, help to solve the vanishing gradient problem;

therefore, allowing the realization of efficient deeper learning

models.

The combination of reinforcement learning and deep neural

networks - known as Deep Reinforcement Learning - has re-

solved several limitations of reinforcement learning including

the limitation in the diversity of application domains, the need

for manual engineering features, and their poor scalability for

high-dimensional state-space domains [9].

A DRL agent observes the environment parameters, takes

actions on the environment, and receives a reward feedback for

each action. The objective of the agent is to maximize its total

future rewards. A deep neural network is used to approximate

In our proposed semi-supervised DRL approach, we

adapted generative deep neural networks (e.g., Variational

Autoencoder-VAE) [10] as the semi-supervised component of

IEEE COMMUNICATIONS MAGAZINE, VOL. X, NO. X, XXXXX 201X 5

Fig. 3. A conceptual structure of a deep reinforcement learning system.

the model. In our proposed model, VAE is extended to produce

the probability of each action in the system.

The proposed approach was applied in a smart campus

project as part of a smart city [11]. The objective of the semi-

supervised learning agent is to provide indoor localization

annotated with the location details (i.e., labeled data).

Figure 4 compares the supervised and semi-supervised DRL

models. The results indicate that using the semi-supervised

DRL model that uses the combination of labeled and unlabeled

data improves the performance of the system. From the total

rewards point of view, the semi-supervised model achieves

higher rewards quickly compared to the supervised model. It

gains between 60% to 100% more rewards compared to the

supervised model gains in different number of epochs. In terms

of accuracy of localization, the semi-supervised model reaches

IoT resource-constrained devices. However, more investigation

is needed to bring this algorithm to IoT devices.

V. SMART CITY USE CASES

utilization of the city’s resources. In addition, providing smart

solutions for transportation, healthcare, convenience, agricul-

ture, and government are the main premises of a smart city.

In this section we present smart city use cases that illustrate

Fig. 4. The performance of semi-supervised DRL model versus supervised

DRL model showing the average total rewards and the average distance to

the target over different epoch counts.

the use of semi-supervised learning to provide better services

to the city’s residents.

A. Water

California has experienced an intense drought period in

prediction of water usage and the available water resources can

help secure water for drought periods. Moreover, monitoring

the level and quality of water in creeks using crowd-sensing

data (e.g., the amount of trash, level of water, picture of trash,

etc. in IBM CreekWatch) along with data from other IoT-based

approaches such as smart water meters can help to achieve

efficient and sustainable water provisioning. In this context,

the images of trash in water can be used by a semi-supervised

DRL system to identify the type of trash automatically and

perform the required action at the location.

IEEE COMMUNICATIONS MAGAZINE, VOL. X, NO. X, XXXXX 201X 6

supervised DRL agent can be utilized and integrated into this

context aiming to keep the optimal power usage by controlling

when to turn appliances on or off. Due to the presence of

many unlabeled data generated by NILM, the performance

supervised DRL agent.

ligence. This is also inline with the security and privacy

preservation requirement since data is not transferred to

the fog or cloud.

• Big dataset shortage: Development and evaluation of

smart city applications need real-world datasets which

IEEE COMMUNICATIONS MAGAZINE, VOL. X, NO. X, XXXXX 201X 7

are not readily available for many application domains.

It is required to confirm results based on simulated big

sets. By changing the distribution, the trained model

needs to be rebuilt from new training data. For example,

in a radio frequency based localization application (e.g.,

WiFi, BLE), the RSSI values for the same time and posi-

tion in Android and iOS devices are different. The trained

localization model on one platform can be transferred to

the new platform without the need to collect RSSI values

for other devices. Transfer Learning is a field of research

that can help in such scenarios [15].

• Integration with semantic technologies is also a need for

and these VOs can learn, decide, and act autonomously.

• Interacting with humans in a natural way is a critical need

for the new generation of smart city systems since citizens

are the main players in smart cities. The small size of

mobile devices and wearables nowadays does not allow

space for touch screens or keyboards. Instead, automatic

speech recognition and natural language understanding is

the most convenient way of interaction with these devices.

VII. CONCLUSION

There are many machine learning algorithms that can be uti-

lized to learn from the big data collected through a smart city’s

infrastructure. However, most traditional machine learning

techniques assume a fixed training model and a static context.

These assumptions do not apply to smart city applications

evolve over the time.

efficient sampling, and devising scalable models. We reviewed

smart city. We proposed a semi-supervised deep reinforcement

[2] P. Kaltenrieder, E. Portmann, and T. Myrach, “Fuzzy knowledge rep-

[3] P. Vlacheas, R. Giaffreda, V. Stavroulaki, D. Kelaidonis, V. Foteinos,

[6] B. Tang, Z. Chen, G. Hefferman, S. Pei, W. Tao, H. He, and Q. Yang,

[7] A. Al-Fuqaha, A. Khreishah, M. Guizani, A. Rayes, and M. Moham-

[8] S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and con-

[11] M. Mohammadi, A. Al-Fuqaha, M. Guizani, and J. S. Oh, “Semi-

[12] T. Boyle, D. Giurco, P. Mukheibir, A. Liu, C. Moy, S. White, and

[13] D. A. Talbot and M. Paz-Canales, “Smart Grid Paybacks: The Chat-