Downloaded 189 times
Uploaded bySimon Guest
Creating Context-Aware Applications
The document discusses workflows, challenges, and opportunities in developing context-aware applications, focusing on the role of various sensors in determining user context. It outlines key steps such as sensor aggregation, data storage, context perception, and user interaction, while also highlighting challenges like context mismatch and user annoyance. Furthermore, it identifies potential applications in retail, automation, and travel, along with the importance of user feedback and triggers in enhancing the user experience.
More Related Content
![[2C3]Developing context-aware applications](https://cdn.slidesharecdn.com/ss_thumbnails/2c3marsalgavalda-expectlabs-final-140929193137-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
![[SOCRS2013]Differential Context Modeling in Collaborative Filtering](https://cdn.slidesharecdn.com/ss_thumbnails/slide2013socrs2013differentialcontextmodelingincollaborativefiltering-130519153030-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Creating Context-Aware Applications
![Governance, Deployment & Methodologies for Agentic Automation [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day2-251112135038-a386476d-thumbnail.jpg?width=640&height=640&fit=bounds)
Creating Context-Aware Applications
- 1. Context-Aware Applications Workflows, Challenges,and Opportunities Simon Guest Distinguished Engineer Neudesic, LLC [email protected] Image Attribution: http://www.flickr.com/photos/48600098314@N01/2612632787 1
- 2. Workflow for DeterminingContext 2
- 3. 1. Determine Sensors SensorsCritical to Determine Context • • • • • Hard Sensors – GPS, Accelerometer, Gyro, Temperature, Humidity, etc. Soft Sensors – Calendar, Facebook Friends, Profile, Likes/Dislikes, History, etc. Mobile Devices – Arrays of sensors, with new ones being introduced all the time (e.g. fingerprint democratized with Android) Sensor Co-processors – Sensors moving to co-processors means lower power consumption (which means that sensors can be on all of the time) Bluetooth LE – Greatly extends the the range and types of sensors (e.g. indoor proximity and awareness) 3
- 4.
- 5. 2. Sensor Aggregation WeDon’t Hard Sensors GPS • Accel. • Temp Noise • Step Need *All* of the Raw Sensor Data to Determine Context Augmented Sensors – Algorithm that augments or abstracts existing sensors (e.g. “Shaking” using the accelerometer) Extended to Motion APIs – Google introducing standing, walking, cycling, driving “sensors” in Android 4.3 Vendor Opportunities – Multiple vendors building SDKs for sensor aggregation – inc. Semusi (Male/Female “sensor” from accelerometer and gyro) Soft Sensors Profile Schedule History Likes/Dis . Social etc. 5
- 6.
- 7. Hard Sensors GPS Collections Accel. Geo Temp Motion Noise Environ. Step etc. Soft Sensors Profile Schedule Events History NewLike Likes/Dis . Friend Social etc. etc. 7
- 8. 3. Send toNon-Relational Data Store Goal is Hard Sensors to capture aggregated sensor data quickly and efficiently Collections • Schema-less– No need to care about schema or structure Accel. Fire and Forget – No need to send confirmation back to client, or handle Geo • Temp missed messages or retries Motion • Scale and Partitions – Data store should be a single entry point that Noise Environ. automatically handles scale and partitioning GPS Step etc. Soft Sensors Profile Schedule Events History New Like Likes/Dis . Friend Social etc. etc. 8
- 9. Hard Sensors GPS Collections Accel. Geo Temp Motion Noise Environ. Step etc. Soft Sensors Profile Schedule Events History NewLike Likes/Dis . Friend Social etc. etc. 9
- 10. Hard Sensors GPS Collections Accel. Geo Temp Motion Noise Environ. Step etc. Non-Relational Store Soft Sensors Profile Schedule Events History NewLike Likes/Dis . Friend Social etc. etc. 10
- 11. 4. Determine PerceivedContext Analyze Hard Sensors data store to determine one or more perceived contexts Collections Simple Matches– Single sensor data can often reveal simple matches (e.g. Accel. movement or in/out geo-fenced location) Geo Non-Relational • Temp Combined Matches – Multiple parallel or serialized sensors to reveal more Motion Store complex matches (e.g. movement with external GPS, combined with loss of Noise signalEnviron.walking activity might indicate parked in underground parking and Step structure). etc. • Learned Matches – Multiple sensor data used to reveal matches through Soft Sensors patterns (e.g. driving detected at same time every day might indicate Profile commuting, which is turn can infer place of work) • Timing – Context often derived from “time blocks” of sensor data – e.g. what is Events Schedule happening now vs. what has happened over the last 30 minutes GPS • History New Like Likes/Dis . Friend Social etc. etc. 11
- 12. Hard Sensors GPS Collections Accel. Geo Temp Motion Noise Environ. Step etc. Non-Relational Store Soft Sensors Profile Schedule Events History NewLike Likes/Dis . Friend Social etc. etc. 12
- 13. Query Hard Sensors GPS Collections Accel. Geo Temp Motion Noise Environ. Step etc. Perceived Context At Work Non-Relational Store Commuting InTraffic Soft Sensors At Home Profile Schedule Events History New Like Likes/Dis . Friend Social etc. etc. Watching TV Asleep Make determination etc. 13
- 14. 5. Triggers onContext Change Triggers Hard Sensors used to monitor change inQuery context Perceived Context Collections Change in Context– Move from one perceived context state to another (e.g. At Work Accel. from in Geo office to commuting) Non-Relational • Temp External Data Source– Triggers often rely on external data source (which can Motion Store Commuting be interpreted as another soft sensor). Could include environmental data, move Noise Environ. listings, weather forecasts, etc. Step Machine Learning– Often benefits from some kind of ML to help determine etc. • In Traffic change in context across multiple sensors, especially in recommendations Soft Sensors space GPS • At Home Profile Schedule Events History New Like Likes/Dis . Friend Social etc. etc. Watching TV Asleep Make determination etc. 14
- 15. Query Hard Sensors GPS Collections Accel. Geo Temp Motion Noise Environ. Step etc. Perceived Context At Work Non-Relational Store Commuting InTraffic Soft Sensors At Home Profile Schedule Events History New Like Likes/Dis . Friend Social etc. etc. Watching TV Asleep Make determination etc. 15
- 16. Query Hard Sensors GPS Collections Accel. Geo Temp Motion Noise etc. Context Change (Triggers) Environ. Step Perceived Context AtWork Non-Relational Store Commuting House too cold In Traffic Soft Sensors At Home Profile Schedule Events History Friend Social etc. Timing Loop Flipping channels Better movie New Like Likes/Dis . Started commute etc. Watching TV Asleep Make determination etc. External Data 16
- 17. 6. Invoke Functions Certain HardSensors Triggers GPS • Accel. Query Used to Invoke Functions Perceived Context Collections Take Action– Functions take action based At Work on triggers. may or Geo not involve some user input. may Temp Motion Noise etc. Commuting House too cold In Traffic Soft Sensors At Home Profile Schedule Events History Friend Social etc. Timing Loop Flipping channels Better movie New Like Likes/Dis . that Started commute Environ. Step Non-Relational Store Context Change Invoke an action (Triggers) etc. Watching TV Asleep Make determination etc. External Data 17
- 18. Query Hard Sensors GPS Collections Accel. Geo Temp Motion Noise etc. Context Change (Triggers) Environ. Step Perceived Context AtWork Non-Relational Store Commuting House too cold In Traffic Soft Sensors At Home Profile Schedule Events History Friend Social etc. Timing Loop Flipping channels Better movie New Like Likes/Dis . Started commute etc. Watching TV Asleep Make determination etc. External Data 18
- 19. Query Hard Sensors GPS Collections Accel. Geo Perceived Context Temp Motion Noise Non-Relational Store Started commute Commuting House too cold Environ. Step AtWork Context Change (Triggers) etc. In Traffic Soft Sensors At Home Profile Schedule Events History Friend Social etc. Flipping channels Better movie Turn on heating Adaptive application Make recommendation Proactive application New Like Likes/Dis . Watching TV Timing Loop Functions etc. Asleep Make determination etc. External Data 19
- 20. 7. User Interactionand Feedback Loop Interact Hard Sensors Perceived with user to notify them ofQuery action, and/or provide feedback any Context GPS Collections Step etc. Context Change function (Triggers) • User Notification– Notify the user of the result from a At Work Accel. User Interaction/Confirmation – Get confirmation from the user that a function Geo • Functions Started shouldMotion happen, Non-Relationalrecommendation. really or provide a commute Temp Store Commuting • Feedback Loop – Provide option for user to submit feedback based on the Turn on heating House too Noise Environ. cold function (like, dislike, rating, etc.) Adaptive application In Traffic Soft Sensors At Home Profile Schedule Events History Friend Social etc. Flipping channels Better movie Make recommendation Proactive application New Like Likes/Dis . Watching TV Timing Loop etc. Asleep Make determination etc. External Data 20
- 21. Query Hard Sensors GPS Collections Accel. Geo Perceived Context Temp Motion Noise Non-Relational Store Started commute Commuting House too cold Environ. Step AtWork Context Change (Triggers) etc. In Traffic Soft Sensors At Home Profile Schedule Events History Friend Social etc. Flipping channels Better movie Turn on heating Adaptive application Make recommendation Proactive application New Like Likes/Dis . Watching TV Timing Loop Functions etc. Asleep Make determination etc. External Data 21
- 22. User Interaction Decision Making Feedback Query Hard Sensors GPS Collections Accel. Geo Perceived Context Temp Motion Noise Non-Relational Store Started commute Commuting Housetoo cold Environ. Step At Work Context Change (Triggers) etc. In Traffic Soft Sensors At Home Profile Schedule Events History Friend Social etc. Flipping channels Better movie Turn on heating Adaptive application Make recommendation Proactive application New Like Likes/Dis . Watching TV Timing Loop Functions etc. Asleep Make determination etc. External Data 22
- 23. Types of Context-AwareApplications 23
- 24. Types of Context-AwareApplications Adaptive • • • Act on behalf of the user Try to adapt to the user’s context Often based using single-trigger • • • Turn on heating when leave for home Phone to silent in meetings Offer music library when get on bus Proactive • • • • Involving the user Often tied to a recommendation engine Often requires user-interaction Often uses multiple triggers • Recommend movie to watch when watching TV Display offer based on history when walking through grocery store • • Need to be careful if building adaptive UI, especially if hiding/showing features • UI needs to be unobtrusive, and easy to dismiss 24
- 25. Challenges When DevelopingContext-Aware Applications 25
- 26. Challenges Context Mismatch • • • Sometimes theperceived context is just wrong… “You really think I want to do this now?!?!” Multiple contexts help accuracy, and feedback mechanism is critical Being Creepy • • • Sharing everything that you know about the context/trigger can appear creepy “Your Mom’s birthday is tomorrow, and you’ve missed the last three years in a row. Do you want to pull into the 7-Eleven ahead? They have flowers on sale…” Reveal only enough to let the user deduce the same context, and feedback again is critical Being Annoying • • • Interruptions should follow normal human behaviors “I wanted to let you know that I’ve turned the heating on” – “Not now, I’m driving!” Knowing the context of the recipient during the UI loop is also important 26
- 27.
- 28. Examples of Opportunities Retail • • Opportunisticproduct offers while shopping Shopping recommendations based on current context Real Estate • Contextual awareness of prospective buyers searching for properties Gaming • Using context to enhance the gaming experience of patrons Field Employees • Context-aware applications that helps field-based employees become more productive Home Automation • Context-aware applications that interact with other systems in the home Travel Applications • More intelligent travel applications through context 28
- 29.
- 30. References • • • Shilt, Adams, Want(1994): Proceedings of the Workshop on Mobile Computing Systems and Applications • http://www.interactiondesign.org/references/conferences/proceedings_of_the_workshop_on_mo bile_computing_systems_and_applications.html Swati A. Sonawane: Context-Aware Computing • http://www.slideshare.net/swatibaiger/context-aware-computing14084995 Albrecht Schmidt (2013): Context-Awareness, Context-Aware User Interfaces, and Implicit Interaction • http://www.interaction-design.org/encyclopedia/contextaware_computing.html 30
- 31. Thank You Simon Guest DistinguishedEngineer Neudesic, LLC [email protected] @simonguest 31