Artificial Intelligence, despite its benefits, presents designers with unprecedented challenges that demand our action.
The responsibility of us designers to protect user interest and value has never been so important. Not only we’re living in a trendy era with the rise of ChatGPT and many other AI tools, but we’re also experiencing how these systems shifts control away from the user. Whether it’s automating tasks, discovering hidden patterns, enhancing user experiences, generating new content, or optimising decisions, AI’s existence is driven by its ability to learn from data and improve over time without explicit programming. Currently, the speed and the way these super powerful technologies are being rolled out, we cannot follow as society, which causes great impact on the economy, on mental health, on culture and on society as whole.
“We, designers, have to own up to our role in this emerging predictive world because we have our invisible hand meddling with the controls”— Helen Armstrong
This quote is from the wonderful book “Big Data, Big Design: Why Designers Should Care About Artificial Intelligence” authored by Helen Armstrong, which explores the intersection of design and artificial intelligence, emphasising the role of designers in shaping AI-driven experiences. In a predictive digital world, we play a crucial role in ensuring that AI solutions are ethical, transparent, and aligned with human values. We must consider the potential biases in data, the impact on user privacy, and the societal implications of our creations. By prioritising responsibility and accountability, designers can help build a future where AI technologies enhance human capabilities and improve quality of life.
The book explores these issues in depth, featuring contributions from various designers, educators, and researchers who work at the intersection of artificial intelligence and design. It serves as my primary reference for the questions and discussions raised in this article, but I have drawn from numerous other sources to provide a comprehensive perspective, all of which are listed at the end of this article.
https://helenarmstrong.info/big-data-big-design/
The central theme of the book — and of this article — is to emphasize the responsibility of designers in the ethical development of AI products and services. Our discipline is fundamentally human-centric, focusing on understanding and addressing human needs and desires within the broader societal context. The psychology, research, and strategy we employ in designing our products have a profound impact on the physical, mental, and emotional well-being of people worldwide.
AI is human-centric…?
Human-centric design, like inclusive and participatory approaches, ensures fairness in data models, mitigates bias, and promotes inclusivity. It is widely recognised that human-centric design has been the guiding principle of all design frameworks and processes over the past decade, and this should be no different when working with AI. These perspectives have been valuable because they inform decision-makers about potential human impacts and help anticipate unintended consequences. Initiatives, such as Stanford University’s Human-Centered AI Institute and MIT’s substantial investment in AI education, exemplify the global effort to prioritise a human-centric approach. According to this article by Sarah Tan, by exploring AI’s human impact through various Human-Centered Design (HCD) approaches, design aligns values between humans and machines, integrating ethics at the project’s core.
Putting humans at the center of AI systems — Human-Centered Artificial Intelligence: Three Fresh Ideas. Shneiderman B. (2020)
Advocating for user needs, design research has the power to examine AI implications in real-world contexts — addressing socio-economic dynamics too. However, this isn’t easy. We, as designers working in the industry, often feel pressured to quickly adopt the latest trends without time for reflection. We are frequently required to use behavioural research to influence user choices, sometimes at the expense of their well-being. Additionally, we must operate in fast-paced environments, adopting to the ‘move fast and break things’ mentality popularised by early Facebook culture. This approach can make it harder for us to design thoughtfully and ethically
But how to stand up for people in the face of such forces?
AI capabilities all designers should know
Here’s what I aim to achieve with this article: to inspire designers to get more involved in creating AI systems, so we can make the process more ethical and human-centered. This means using design thinking as our guide to reduce the negative impacts of AI on society by developing human-centered AI systems and guidelines.
How we can achieve this is still being figured out, and we are all part of this process! To help, I want to highlight the main capabilities discussed in Armstrong’s book, where we designers should be actively involved. We need to advocate for design decisions that meet user needs and respect the socio-economic context in an ethical way. It won’t be easy, but we can start paving the way with these elements.
1. Designing to power up predictions
Prediction is a central capability of machine learning (a subset of AI that involves training algorithms on data to make predictions or decisions without being explicitly programmed). In the context of predictive modelling, the goal of machine learning is to forecast future events or outcomes and anticipate behaviours based on patterns found in historical data.
For instance, we can think of Gmail’s feature that autocompletes sentences based on your past behaviour. Over time, the system learns an individual’s style so well that it can finish sentences and even suggest appropriate tones or sentiments.
Screen capture from my personal Gmail
Or we can think of the numerous applications across various industries, such as stock price forecasting, inventory and supply chain optimisation, healthcare predictions (such as disease outbreak forecasting), targeted marketing campaigns, and other personalised user experiences.
While these AI-driven predictions aren’t always 100% accurate, they offer significant benefits to businesses. The advantage is that by anticipating future demands, trends, and needs, companies can better prepare for opportunities and gain a competitive edge. For that end, designers are required to plan for a shifting experiential landscape when working with intelligent technology in their projects.
On the other hand, designers cannot blindly apply AI technology without the risk of subjecting humans to discrimination, surveillance, and/or manipulation, not just individually but at scale. Predictive models can sometimes reinforce existing biases or lead to unintended consequences.
“Face Cages are a dramatization of the abstract violence of the biometric diagram”. More on: https://zachblas.info/works/face-cages/
Designers have a responsibility to foresee these potential issues and take steps to mitigate them, ensuring that their designs are ethical and inclusive. As Kate Crawford, AI Now Institute founder, puts is:
Understanding the politics within AI systems matters more than ever, as they are quickly moving into the architecture of social institutions: deciding whom to interview for a job, which students are playing attention in class, which suspects to arrest, and much else.
2. Designing to anticipate future scenarios
In her book, Helen Armstrong explores the concept of anticipation, showing how designers can use AI to predict and meet user needs proactively. This involves leveraging data-driven insights to stay one step ahead of users, reducing friction and enhancing the overall user experience.
One advantage of anticipatory design is that it reduces decision fatigue by minimizing the cognitive load on users through predictive systems. These systems offer relevant choices or take actions on behalf of users. For example, companies like Netflix and Amazon use predictive analytics to recommend products or content. However, there are ethical implications, as such systems might infringe on user autonomy or privacy. As Armstrong rightfully states on page 210:
The promise of anticipatory design lies in its potential to craft experiences that feel intuitive and personalised. Yet, with this power comes the duty to wield it responsibly, ensuring that the human element remains at the heart of every design decision.
Fortunately, there is a forward-thinking approach that allows us to create products and services that are resilient to change and can evolve with user expectations. Designers can use AI to simulate and model potential future scenarios, anticipating various user needs, environmental factors, and societal changes. This ensures that our designs remain relevant and adaptable.
Flock Defines Flight Risk to Make Insurance for Drone Operations Simple: the insurance company worked with studio IF to create an interface that help clients understand how automated decisions were made around insurance rates.
3. Designing to create personal experiences
Personalisation in user experience design involves tailoring interfaces, content, and interactions to meet the specific needs, preferences, and contexts of individual users. When we talk about personalisation in the context of AI, it often means leveraging predictive technologies and data analysis to deliver experiences that are more relevant and valuable to users at any given moment. Personalisation ensures that the information or actions presented to the user are useful, usable, and desirable, inspired by the popular quote from design researcher Liz Sanders.
AI can enhance personalisation by analysing vast amounts of user data to identify patterns and preferences, using predictive models to anticipate what a user might need or want next — this can involve considering the user’s current context, such as location or time of day, to provide more relevant suggestions. However, personalisation also raises ethical concerns, including privacy, bias, and transparency. This is why we, designers, play a crucial role in ensuring that personalisation is implemented effectively and ethically. We must understand user needs and preferences through research and testing, and continuously gather feedback to refine personalisation features.
https://medium.com/media/6cf6ef6eec5b59e000dc098865a8e440/href
We can say that personalisation is a powerful tool in a designer’s toolkit, but as Armstrong emphasizes, the success of personalised experiences hinges on the thoughtful and ethical application of AI insights. It is crucial to always prioritise the user’s best interests, ensuring that personalisation efforts are not only effective but also respectful and beneficial to the individual.
4. Using AI as a Design Material
Designing for AI is challenging because the technology is invisible and abstract. As designers, if we can’t ‘sketch’ a solution, it feels like losing a crucial ability. As interaction design professor Philip Van Allen puts it, ‘it’s like our arm is cut off.’ To address this, he created a no-code programming environment for his students, providing easy access to machine learning. The tool is called Delft AI Toolkit and it simulates an AI system in 3D, allowing designers to observe and manipulate the system’s behaviour and data sources in a virtual space before investing time and expertise into model training and building a physical robot.
https://medium.com/media/7554deb21a70eb4fc12cd2a0689fdd80/href
Another approach to working with AI as a design material comes from John Zimmerman, a professor at Carnegie Mellon University. He developed a method that focuses on the AI system capabilities rather than the underlying technology. Zimmerman observes that many of his design students are uncertain of what’s possible when working with AI technologies, so he uses a “match-making” system which helps them grasp the AI system’s potential. For example, he presents a two-class text classifier and asks, ‘What could you do with this, and for whom?’. This method helps designers identify existing capabilities and apply them in innovative ways. As with any design tool, the more we experiment with it, the more comfortable we become, and the more examples and abstractions of AI capabilities we will generate.
Janelle Shane’s experiments with DALL-E 3 playfully tease out all the ways algorithms can “get things wrong”. Here, she jumps into the algorithmic training process to generate candy heart messages. More on: https://www.aiweirdness.com/dall-e3-generates-candy-hearts/
A third approach, which I find particularly effective, is using the Object-Oriented UX (OOUX) methodology. OOUX focuses on ‘objects’ before ‘actions,’ aligning with how users perceive the real world. It borrows concepts from object-oriented programming but applies them to user experience design. This approach works well with AI workflows by organising data and interactions around objects, making complex AI systems more intuitive for non-developers by matching their mental models. Sophia Prater is a leading expert in the OOUX field, and I highly recommend her materials and podcasts on the subject.
On a personal note, I recently completed my first project using this approach for a native AI product. The methodology was incredibly helpful. It allowed me to visualise the system’s functionality and limitations more clearly, which also aided the engineers. Additionally, it made my design process more tangible through object mapping and user flows, enhancing the quality and accuracy of my proposed solutions.
A beautifully chaotic collaboration between me and the engineering team using the OOUX approachML is the new UX. I envision UX practitioners leveraging machine learning as a design material creatively and thoughtfully, guiding users and technologists toward a deliberative ML-mediated future — Qian Yang, Cornell University
5. Designing to reduce climate impact of AI
Although this isn’t mentioned in the book and isn’t a ‘AI capability,’ it is certainly one of the key skills designers should possess, especially in the 21st century with the rise of AI technologies.
There’s an article called “How to Design Climate-Forward AI Companies” by Santhi Analytis that emphasises the important role designers play in mitigating the environmental impact of AI technologies. It frames the challenge of AI’s growing carbon footprint — fueled by energy-intensive processes like model training and data center operations — as one that requires urgent attention from both engineers and designers. Designers can contribute by creating energy-efficient UI/UX elements, minimizing unnecessary data and media loads, and applying green software principles. Additionally, designers can push for socially equitable uses of AI by developing inclusive, climate-conscious user experiences.
The Intercept Brasil (in Portuguese) also published last year an article stating that the major tech companies like Microsoft, Google, and Amazon are now turning to nuclear energy as a solution in response to the growing energy demands of AI and data centers. Microsoft plans to reopen the Three Mile Island plant, Google has partnered with Kairos Power for small reactors, and Amazon has invested $500 million in nuclear technology. Many of these data centers are being built in countries from the Global South, where labor and infrastructure costs are lower, raising concerns about resource exploitation. While nuclear energy is carbon-free and efficient, it is costly and carries risks, and critics argue that “big tech” is addressing a crisis of its own making, prioritizing rapid technological expansion over sustainability.
The industry’s “move fast” mentality raises concerns about whether nuclear energy can truly keep pace with AI’s accelerating energy consumption, or if it will simply become another chapter in big tech’s history of unchecked growth. This Vox documentary tries to answer this question:
https://medium.com/media/8d8c5a33bba53566072d960d2be0f0f4/href
In this other article from MIT, the author Andrew Winston also highlights the growing environmental impact of AI and provides recommendations to reduce its carbon footprint, such as using existing models instead of creating new ones. It emphasises the role of designers in this issue, urging them to develop AI expertise to simplify complex concepts, communicate the “nutriscore” of AI in their products (its usage and environmental cost), and adhere to an ethical “code of conduct” similar to the EU’s AI Act, ensuring AI products are safe, transparent, traceable, non-discriminatory, and environmentally responsible.
As designers, we can influence user behaviour and company policies by incorporating transparent data models that reflect the true costs of AI, thus ensuring the technology not only solves human problems but also helps tackle environmental challenges.
Conclusion: Let’s take action
It’s easy to feel overwhelmed by the urgent need for AI ethics. AI is a versatile and dynamic field, capable of performing a wide range of tasks that can help solve real-world problems. However, algorithmic systems often reflect and amplify existing societal biases. Until interfaces clearly communicate the logic behind these algorithmic decisions, users won’t be able to hold these systems accountable. This is why designers and the communities they serve need to understand digital rights. We must hold ourselves and the industry accountable for the choices we make through our designs.
The key question is: Will we allow machine learning to prey on those already victimised by society, or will we use this technology as a mechanism for equity and justice?
Here are some actions we can start taking as designers in this industry:
✅ Design for anticipation to prepare for multiple possible futures, making our designs more flexible and future-proof✅ Think critically about the long-term impacts of our designs and the data-driven decisions that underpin them✅ Articulate digital rights and guide users with transparency toward options through design interactions✅ Break down complex privacy agreements into quick, comprehensible, just-in-time interactions✅ Prototype future-facing concepts, like personal privacy agents✅ Support collective digital rights organisations and community-driven dataset initiatives✅ Ask early questions about ethics and biases to ensure technology adapts to human needs, not the other way around
How do you approach building AI products? What examples do you know of ethical and inclusive design solutions?
References
“Big Data, Big Design: Why Designers Should Care About Artificial Intelligence” — Helen Armstrong (Book)“Design and the Question of History” — Tony Fry, Clive Dilnot, Susan Stewart (Book)“Human-Centered Design for AI” — Webinar by Niwal Sheikh (Product Design Lead, Netflix) by IxDF“How to Design Climate-Forward AI Companies” — Article by Santhi Analytis“The problem with AI development today: Designers need to step up” — Article by Sarah Tan“Will AI Help or Hurt Sustainability? Yes” — Article by Andrew Winston“What is predictive AI?” — Article by Cloudflare“Reviewing the Terms & Conditions of popular generative AI tools” — Article by David Serrault“The Problem with AI Development: Designers need to step up” — Article by Sarah Tan“Design For AI (Artificial Intelligence)” — Article by Sudarshan Sahu“Design Against AI” — Website by John Maeda“The Intercept: ‘Big techs apelam para energia nuclear’” — Article from Portal InvestNE mentioning the original piece from The Intercept Brasil“OOUX” — Website by Sophia Prater“AI Now Institute” — Website
Why AI (desperately) needs designers was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
