BIXBY- Samsung’s Intelligent Agent

Client: Samsung R&D Institute, Bangalore & South Korea.
Project Timeline: 1.5 Years (Jan 2016- July 2017)
Project MangerBiju Mathew Neyyan, Chief Designer (at the time), UX Labs, Samsung R&D Institute, Bangalore
Team MembersArjun Raj Kumar, Sriram Shashank
Design Keywords: Intelligence, Contextual awareness, Voice Interface Design, Text Interface Design for a AI Agent.

 

Note:
This project and employment was under Samsung’s NDA.
Hence, not much internal confidential information can be presented in this portfolio, but will clarify what my role was and we can have a chat about it.

design brief

A new way to interact with your phone. Bixby knows your phone’s apps and how to use them.Plus, you can talk, tap or type, because Bixby understands it all. I was involved in design iterations of voice interface, creating principles for typing interface with an agent and suggesting overall improvement of basic interactions with the agent. Lately, I have been a part of testing of utterances and feasibility.

 

my role

 

1. Concept generation (Design Ideation) of intelligent agent based on Time Management, iterations of the voice assistant and chat interface (led this section in the later parts of the projects) for Bixby. Additionally, I was involved in concept generation for chat interface of Bixby for DEX mode. Generation of UX and UI guidelines and documents for daily prototyping into GUI and active prototypes.

3. Coordination and collaboration (Design Implementation and Strategy) with HQ (South Korea) to represent Bangalore team for the project. Additionally, this involved collaboration and active discussions with business and software engineering teams to understand the back-end of Bixby's architecture, NLP structures as impacting UX design decisions, and translating that knowledge into design production; and vice-versa.

4. “Water-proofing” (Design Iteration) of implemented Bixby Concepts to iterate and give alternative solutions of interactions and concepts (involved quick prototyping and making of high fidelity screens for better presentation). Evaluating VUI models created for Bixby use-cases.

5. Usability Testing (Design Evaluation) of Bixby in daily-usage, recording issues, constant re-building of design concepts, and follow ups with technical teams.

reflection on experience

 

This project has built the design practitioner skills in me to handle the stakeholder and user side of designing a novel product that never existed in the market till that date. Bixby was called the Samsung’s intelligent agent, but the clarity and marketing was difficult as it was seen as another Google Assistant or Siri. the truth is, it is not. Bixby was a revolution in the UI interaction on mobile devices to control your touch interaction through simple and natural voice commands. This takes more than creating a single interface for an assistant and building the NLP structure in the back-end like traditional conversational user interfaces. Given this novel aspect, there were challenges in multiple directions in terms of expert’s knowledge to design and construct such interfaces AND convey this complexity through “best experience” for users. In the beginning, there were no users, but customers as we had to sell the idea of this product. Therefore, it began with multiple avenues such as designing the basic agent’s personality (voice, type and vision), VUIs for Samsung’s connected devices, Bixby on multiple device landscapes and expanding future capabilities through DEX modes. The project had multiple goals to make Bixby the face of Samsung’s products. This is when I understood how business works and its impact on designing the complex ecosystems we create through these products. This is learning 1 !

Learning 2 was actually dealing with stakeholders of different cultural, disciplinary and expert backgrounds. I was tagged a “designer”, sometimes “researcher” in through this whole process, but suddenly I saw myself representing my home team in Headquarters. I had to interface with product managers, developers, VPs and fellow designers of other units. The conversations explode the possibilities, feasibility and constraints you need to face. These had to be translated into your design decisions, sometimes even if you are uncomfortable or hope for something more than what is accepted.

There was a rapid phase of my growth through these experiences, culturally, personally and professionally. There were multiple dimensions of cross-cultural interactions and stakeholder discussions that helped me to “become a designer” that I am today that helps me. The experiences you live affect the kind of designer you turn out to be and this inter-dependency has to be reflected upon constantly as designers have more responsibilities than just creating usable products. This is my learning 3!

important articles

VP Injong Rhee note on Samsung BIXBY:

Samsung’s blog on Bixby’s capabilities:

Next
Next

S-Health- Gamification for Fitness Motivation