Today at Lenovo Tech World, we demonstrated what the next decade of smart technology will look like in key areas of society, including manufacturing, healthcare, the workplace and the classroom. We unveiled several concept products and demonstrated how emerging technology will shape nearly everything we do. We build concepts like AI-powered robots to show what’s possible and to uncover and break through barriers – to do the challenging technical and often unseen work that makes innovation happen. The more we push ourselves, the faster we can pilot technologies and bring the right ones to market. Let’s look at some of the key innovations from the show.
The End-to-End Tech Impact in Manufacturing
From interactive robots to augmented reality glasses to computer vision, we have integrated solutions and game-changing devices that reshape the way manufacturing work gets done.
Assistive Robots, Smart Glasses and Integrated Solutions
Lenovo designed the Daystar Robot to interact in real time in a remote environment by using the latest technologies including telepresence, teleoperation, and teaching capabilities. Here’s how it works: The human interacts with the robot via a 3D video stream from the Daystar augmented reality (AR) headset using a 5G network for real-time transmission of data. The new G2 lightweight Daystar AR glasses can be connected to our external computing pack or a PC and a smartphone. The user’s head position controls the robot’s arm, and the handheld device controls the end effector of the robot to precisely control its movements. The robot’s camera moves along the same angle as the human user to make this experience dynamic and life-like with a fast response time.
We also equipped it with collision detection technology similar to vehicle systems to allow safe operation in a manufacturing environment. Using a teacher-student model, once the robot learns a task, no reteaching is required. The Daystar Robot offers a new choice for operational efficiency and increased safety for use in restricted or dangerous workspaces, accelerating and automating assembly lines, painting, inspections, and more.
We’ve packaged these computer vision and robotics technologies to create our Machine Vision Solution for Smart Manufacturing to inspect assembly line defects. In fact, we use it in our laptop and smart phone assembly lines. Previously, human workers performed this task, but now using the Machine Vision Solution for Smart Manufacturing Solution we’re able to reduce worker fatigue and the subjectivity of human judgment while increasing product quality.
To further improve assembly defect detection, we’ve developed Esense, an automatic optical inspection (AOI) solution using computer vision technologies. Used in our Wuhan manufacturing plant, it makes it possible to accomplish the difficult job of detecting inner defects in mobile phones. Esense can identify whether screws and sponges are missing during assembly and whether key components are installed correctly. The solution incorporates computer vision and deep learning technologies and only needs a small sample set to train the algorithm, important when production lines have multiple product models to assemble.
Smart Packing
We designed the AI-enabled Smart Packing Solution to integrate warehousing and logistic processes and networks to improve outbound warehousing, packing efficiency, and volume penalty for shipping orders. The solution aims to solve the challenges in traditional outbound packing processes: unreasonable quantities of combinations of goods to be shipped, inefficient packing arrangements, and sub-optimal dispatching of goods. These challenges can lead to penalties for having high volumes, redundant pallets, or container space waste, resulting in high shipping costs.
The Smart Packing Solution supports human packing by automatically generating an optimal packing plan. It optimizes packing plans based on the information of warehousing goods and optimized packing combinations. As a result, it boosts outbound operations and saves shipping costs, while improving the efficiency and achieving an optimal packing and loading performance.
Smart Production Planning
Manufacturing production planning challenges typically involve long time-cost horizons, low efficiency, and difficulty in balancing multiple objectives. To help solve some of these Lenovo has create a Smart Production Planning Solution. It uses a variety of artificial intelligence (AI) technologies and mathematical optimization algorithms to provide efficient and high-quality production resource allocation solutions in the production line. It focuses on making production planning:
- Real-time, and optimized for multi-objectives,
- Flexible management of tasks,
- Visual management,
- Resource utilization improvement.
A Smarter Supply Chain
The Smart Service Supply Chain Platform integrates forecasting, procurement, and scheduling to result in a more accurate, faster response speed, and automation of each of the modules. This leads to meeting customer needs more quickly, improved customer satisfaction, and reduced operating costs for procurement, transportation, and inventory.
The Smart Service Supply Chain Platform features:
- Fast decision simulation: Real-time service supply chain planning through efficient simulation in complex business scenarios
- Accurate prediction results: Discovering and utilizing hidden patterns in multi-source data, including consumption history, installed base, parts substitution relationship, temperature, and social media events, which are rarely used in manual planning.
- In-depth business insights: Helping planners to make better demand forecasting of new parts in the introduction stage by visualizing the relationship among parts, products, and suppliers with the help of knowledge graph.
- Visual demand forecasting display: Helping planners perform key tasks more confidently and efficiently through the visual and detailed display of demand forecasting
End-to-End With AI-Enabled Customer Service Solution
From manufacturing to supply chain to customer service, Lenovo is creating end-to-end solutions with Moli, an AI solution for large businesses providing customer service. Moli integrates an intelligent customer service robot with human customer service agents to provide personalized services in the manner and language users need at any time and place. Lenovo uses Moli for its global mobile phone and PC product services. Moli also has applicability to human resources, government services, and hospital information desk services as well. The Moli smart services include:
- Deep natural language understanding
- Policy-adaptive dialog manager
- Personalized and flexible natural language generation
- Large-scale knowledge graph building
Healthcare Platforms Transformed
Monitoring System
The Lenovo Smart Health Monitoring System prototype uses AI technology to detect human vital signs, such as heart rate, from videos taken of human faces generated on their smart devices, which could play a role in helping improve personal health monitoring during COVID-19. Signals containing the pulse information are extracted from the color image sequence of a face captured by an ordinary RGB camera. Then the AI performs a signal analysis and calculates heart rate value.
With this system, health tracking is non-invasive, non-contact, easy to operate, and has no extra hardware cost. Meanwhile, the system’s heart rate readings are highly consistent with those from the standard pulse oximeter and therefore, would have great potential in helping popularize personal health tracking on a daily basis. Primarily used for heart rate measurement now, in the future, we plan for it to be able to monitor other vital signs, like respiratory rate, blood pressure, blood oxygen saturation, and stress index.
Contingency & Emergency Care Platform
Lenovo integrated a portfolio of emerging technologies for the smart automated external defibrillator (AED)-enabled Contingency and Emergency Care Platform. This platform integrates cloud computing, communications and storage technologies, as well as technologies like the Internet of Things, AI, blockchain and AR/VR. Indoors, an in-house developed, intelligent navigation service is in place, while outdoors, a third-party map navigation is activated. A contingency commanding “pod” is in charge of emergency care operations and connects to urban contingency and emergency systems. The system can detect people collapsing on the ground and quickly call an ambulance. At the same time, Lenovo volunteers are notified of the emergency incident, who, with the help of the map provided by the Lenovo AED mini-app, would rush to the scene and perform first aid before the ambulance arrives. Big data and AI also help optimize first-aid processes and improve the patient experience.
IoT Integrated & Smart ECG
The Integrated Healthcare IoT Solution focuses on services to government, hospitals, families, and home-based senior care through algorithms, big data analytics, and Internet of Things (IoT) technologies. The solution packages smart devices, a data monitoring cloud platform, and professional healthcare services to manage regional chronic diseases: IoT technology provides real-time ECG and health condition analysis, improves physicians’ efficiency, and decreases healthcare costs. It also helps healthcare authorities collect and analyze data of users for total lifecycle healthcare management of patients. For home-based healthcare, users are able to monitor and record their vital signs outside of the hospital and detect anomalies as early as possible, improving the level of home health intervention and self-health management capabilities.
The ECG Monitoring Sports Patch provides comprehensive ECG and AI-enabled health monitoring services to users. With it, users can conduct pre-workout physical fitness assessment and create reasonable training programs based on the assessment. While doing sports, users can use the patch to monitor dynamic ECG and track their health status in real time. The patch can also analyze users’ health status after a workout and help them with post-workout recovery. In brief, it manages users’ fitness intelligently and helps them achieve reasonable and safe training/workout goals. The Clinical Level ECG Patch provides 24 continuous hours of ECG monitoring service. It is enabled by medical-level ECG monitoring technology and algorithms and is capable of analyzing 28 types of cardiac diseases. The Clinical Level Wearable 3-Lead ECG Belt is capable of recording 72 continuous hours of ECG data in real time, providing professional 3-lead ECG monitoring and data analysis services to users in and outside of the hospital.
Smarter Workplace with New Diarization & Voice Tools
Speaker Diarization for Smart Meeting uses high-performance automatic speech recognition (ASR) and speaker diarization technologies to automatically transcribe meeting audio streams into readable and structured meeting minutes and identity who speaks when. We want to make voice a primary input method equal to that of a mouse and keyboard. Users will be able to have task-based interaction by transforming multi-step, multi-menu, and multi-click operations into a single voice command and will support the following operations:
- Window Management
- Multi-window layouts
- Launch of applications and Settings pages
- Window manipulation: position, snap, maximize, minimize
- Device Control and Settings
- Multi-setting profiles, or
- Individual control of: Speaker, Microphone, Camera, Bluetooth, Performance mode, Airplane Mode, and more
- Application Control
- Stylus tools like pen color, eraser, lasso tool, and more
- Dictation of symbols and short phrases (another application mode enables long-form dictation input as well)
Classroom of the Future
With more children learning remotely than ever before, the need for an in-person experience in a digital format has never been more important. Lenovo developed a Future Classroom project, which aims to impact teaching trends of the future through an interactive, intelligent, immersive, and holographic experience of teaching and learning. The project consists of two parts: Immersive Learning and Telepresence Teaching.
Immersive Learning applies interactive holographic technology to a “Knowledge Forest”, which presents the knowledge architecture in the form of a “forest” in a holographic, 3D way. It also enables users to explore the “forest” in a natural, interactive way. In addition, services like knowledge path navigation and virtual experimenting provided by the system can trigger students’ desire for knowledge and motivate them to learn.
Leveraging panoramic display, an 8K camera as well as holographic projection technologies, Telepresence Teaching enables an immersive online education environment where the teacher can see students in a remote classroom in a panoramic way. The students can see the remote teacher in a holographic way, as if they were in the same classroom. The teaching content will also be showed in a holographic, 3D way and combined with the teacher, these will produce a mixed reality effect.
Infrastructure Upscaled
5G & MEC-Based AR/VR Application
AR / VR technology is being applied to industrial production, design, training, gaming and other fields to provide immersive, virtual visual experiences. However, the current AR / VR terminals generally use local rendering. Due to the limitations of the storage and computing capacity at the end side, they are unable to support the display of high-precision, large-scale 3D virtual scenarios. As an edge computing platform, the MEC platform is based on a lightweight virtualization architecture integrating software and hardware. Its features and functions include: lightweight virtualization as the basic resource service of computing and storage fusion driven by kernel; core technical capabilities such as fast access, flexible arrangement, flexible resource management and support for applications, as well as interactive frameworks of edge cluster and cloud center; flexible business launch, simplified operation and maintenance, and efficient utilization of resources.
Server Memory Expansion Technology
Lenovo has investigated and developed a computer system architecture and prototype that expands memory beyond directly attached DIMMs, leveraging newly developed industry standard interfaces.
High volume rack servers based on Intel or AMD x86 CPUs are continuing to grow more capable to perform computation on large data sets and host database accessible to greater numbers of users. This system capability has especially grown with the number of cores in a single CPU, application accelerators like GPUs and FPGAs, scale-out NVMe storage and high-performance IO. However, it is becoming increasingly difficult to support additional memory in systems, both due to physical space close to the CPU and the large number of pins on the CPU required to add more DDR memory interfaces. There are also physics limitations on the DDR interface that require a tradeoff between higher frequency (and greater bandwidth access to the data in memory) versus expanding the system memory capacity with more modules on those interfaces.
This project addresses these problems with a new system architecture that expands memory on a new low-latency interface called CCIX. This new serial-link interface and protocol leverages the same pins, physical connections and hardware as PCIe interface and can be used interchangeably with PCIe protocol to the same device. The project uses this capability to attach memory on the CCIX interface, ultimately in the same slots as PCIe devices such as NVMe solid state drives in the EDSFF E1.S form factor. Thus, memory resources can be expanded without impact to existing DDR-attached memory along with additional bandwidth to provide access to the gained capacity.
Smart Training for Athletes
In ball games, the biggest pain point of traditional training lies with data that’s not easy to save and share. In response, we created a Smart Training Center Solution for games such as volleyball, whose features include training reviews, livestreaming and data visualization, etc. The solution, equipped with AI video analysis algorithms and intelligent interactive technologies, solves problems with traditional training methods and optimizes training quality. The solution provides personalized imagery analysis to improve training efficiency, dynamic data display and interactive functions to facilitate accurate training reviews, customized camera matrix deployment to achieve seamless observation and comparative analysis, and flexible multi-party information linkage technology to ensure efficient transmission of training information.
We adopted optimized video processing algorithms and high-speed computing and multi-model collaborative inferencing capabilities to solve the difficulties in video processing and analytics when players are in high-speed movement, overlapped or obstructed from view. Also, the solution uses facial recognition to automatically edit video clips of relevant players, which helps the coach to carry out targeted reviews; at the same time, the post estimation algorism, which enables human skeletal joints mapping and pose angle labeling, assists the coach in optimizing players’ poses and movements and therefore, improve training effectiveness and efficiency.
