NTU IoX Center

Augmented Collective Beings 人機共生之感知關鍵技術

In an increasingly connected & automated world, humans are confronted with the challenge of digesting growing volumes of data generated that far outpace the perceptual bandwidth of human biological sensors (i.e., sight, hearing, touch, smell, and taste) and far exceed the computational capacity of human processors (i.e., brains).
Against this backdrop, we aim to create demonstrable technology for Augmented Collective Beings (ACB) that places people into the interaction loop together with intelligent automated machines. Instead of focusing on augmented machines (e.g. robotics or automation as in Industry 4.0), the ACB technology vision is focused on augmented humans with super-sensing, super-cognitive, and super-communicative capabilities. These enhanced capabilities will allow for natural and seamless interaction in-the-loop with smart machines and other agents.

Smart Factory:

Real-Time Monitoring/Tracking of Human, Equipment and Infrastructure

We investigate innovative IoT sensing and communication technologies that can be used in future smart space such as factories, workplaces and buildings. Smart factories are indoor spaces that have many interaction scenarios, including human-human, human-machine, machine-machine, and device-system. The NTU IoX center is especially interested in new technologies that can augment human’s sensing and capabilities using different sensing modalities as well as visual and sensing devices that can be installed on intelligent robots. The technologies produced in this cluster are designed to be easy-to-use and human-friendly.

Smart Task Space:

Multi-Modal Sensing and Feedback for Knowledge Transfer

Focus on the sensing and feedback mechanisms for knowledge transfer using multiple modalities, in an effort to capture users’ nuanced actions and to tailor the feedbacks that suit their immediate needs during work. Projects in this cluster aim to enhance the knowledge transfer between experts and novices by (1) acquiring and constructing expertise using multiple data sources with machine learning, (2) offering tailored feedback and answers to novices’ questions based on real-time image processing as well as intent classification and sentiment analysis, and (3) enriching the interaction among users, knowledge base, and task objects by integrating visual cues of lighting patterns as well as RFID-based haptic mechanism.

SIG

Sensing

SIG-Sensing covers physical sensors, personal interface devices, and the personal interface device platform. Research challenges of interest are sensors, sensor platforms, and sensor data processing units that are low-power, reconfigurable, flexible, and extensible. The research issues of interconnect sensors and devices are considered in this SIG. The challenges of the networking issues are body-area networks and scalable security protocols. The challenges of system-level security and privacy protocols and load balancing mechanisms are addressed in this SIG.

SIG

Learning

SIG-Learning focuses on the processing and handling of various types (visual, audio, bio-signals, etc.) and quantities (e.g., big data) of sensor data. The challenges involve adaptive machine learning techniques that turn low-level data into high-level knowledge that is then used to anticipate future changes in the environment.

SIG

Interaction

SIG-Interaction involves the design of various personal and AR/VR interfaces that interact naturally with humans. The applications that leverage and integrate technologies from SIG-Learning and SIG-Sensing to build ACB applications.

Scenario Tech Group	Smart Task Space	Smart Factory
Tech Group Scenario Interaction SIG	Smart Task Space Social Computing: Supporting Grounding in Communication with IoT-based Knowledge Sharing and Learning Midas Touch: Finger-Worn Interaction Device for Connecting the Connected Things	Smart Factory Natural and Seamless Interaction
Tech Group Scenario Learning SIG	Smart Task Space Interactive Dialogue Learning Learning in Compressed Domain Development of Light Learning Engine in Fog Computing Environments InterActive Machine Learning	Smart Factory
Tech Group Scenario Sensing SIG	Smart Task Space	Smart Factory DynaCollect Enhancing Security and Privacy in Augmented Collective Beings Next-Generation CamCom Distributed Intelligence

Scenario Tech Group	Interaction SIG	Learning SIG	Sensing SIG
Scenario Tech Group Smart Task Space	Interaction SIG Social Computing: Supporting Grounding in Communication with IoT-based Knowledge Sharing and Learning Midas Touch: Finger-Worn Interaction Device for Connecting the Connected Things	Learning SIG Interactive Dialogue Learning Learning in Compressed Domain Development of Light Learning Engine in Fog Computing Environments InterActive Machine Learning	Sensing SIG
Scenario Tech Group Smart Factory	Interaction SIG Natural and Seamless Interaction	Learning SIG	Sensing SIG DynaCollect Enhancing Security and Privacy in Augmented Collective Beings Next-Generation CamCom Distributed Intelligence