Salesforce AI Research

Pixel Playground

Making a complex generative AI system feel simple

Pixel Playground is an experimental multimodal AI platform developed by Salesforce AI Research. I designed an interface that brought image generation, editing, object removal, stylization, and other capabilities into a single conversational experience.

My role

UI design
Information architecture

Impact

Pixel Playground was showcased at Dreamforce '23
Contributed to image analysis capabilities later incorporated into Salesforce products

An exploration into generative AI

Salesforce AI Research was experimenting with new ways to generate and manipulate images using natural language. Pixel Playground was created in order to give researchers a way to test and showcase those capabilities directly.

As Pixel Playground's capabilities grew, however, so too did the complexity of the experience. The platform's vast array of editing functions created a fundamental design challenge:

How do you make a complex set of AI tools feel like one coherent product?

My first instinct was to organize the complexity

The initial iterations focused on giving each capability its own interface. Basic image generation could remain conversational, while advanced editing could use a workspace designed specifically for manipulating an existing image.

This approach worked for simpler features, but became difficult to scale as new and more complex capabilities were added. The interface needed to be flexible enough to grow with the platform, without becoming more complicated.

Then I consolidated capabilities into focused experiences

I decided to group the platform's features into three categories, each with an interface tailored to its functionality. Image-to-text, for example, used a two-column layout that let users converse with the LLM while seeing their edits in real time.

This approach simplified navigation and gave new features a clear place to live, but internal testing revealed another problem: each interface came with its own learning curve. First-time users had to spend time and mental effort familiarizing themselves with each page, creating unnecessary friction during onboarding.

So I removed the categories altogether

Rather than separating capabilities across different categories and interfaces, I consolidated them into a single, flexible conversational workspace that adapted to the task at hand. Individual capabilities were moved into the prompt itself, where users could invoke them with “/”.

One interface, all the capabilities.

A simpler interface required more thoughtful interactions

With the new structure established, I mapped flows for each of the model’s capabilities to define how they would work within the unified experience.

These flows, such as the Quick Edit feature below, helped communicate the behavior of each interaction to the development team.

Finally, I created an instructional guide with visual examples to better describe the platform's abilities to users.

Mobile

With the desktop version completed, I formatted the interface to be compatible on mobile devices using Pixel Playground in a web browser. This would allow attendees at the upcoming Dreamforce expo to experiment with the platform following the presentation.

Result

The completed prototype was presented at Dreamforce 2023, in which attendees were encouraged to download the mobile version to interact with it themselves.

While the project was only initially created to demonstrate the capabilities of Salesforce's in-house LLM and deep learning library, the advancements the project made in computer vision and image analysis contributed to the development of Agentforce, Salesforce's autonomous agent platform which was introduced the following year.