UI workshop to accompany the ‘Intelligent knowledge database – integration of Aleph Alpha into editorial tasks’ AI project

The ‘Intelligent knowledge database – support during Aleph Alpha integration’ project focuses on the use of Aleph Alpha’s large language models to make the day-to-day work involved in maintaining text documents and editorial processes more intelligent and efficient. adesso implements the work processes for data analysis and clustering in the database as a pro bono service. This is done as part of an agile project strategy, with utopia covering the infrastructure costs after implementation. A core goal of the project is to ensure the prudent use of resources, particularly with regard to the compute needed to run large language models.

In my blog post, I will provide a detailed look inside the workshop that accompanies the AI project as well as key aspects of it.

The EPIC 2 workshop with utopia

EPIC 2 focuses on the integration of Aleph Alpha Luminous as a way to optimise the search for relevant content. Three central user stories were identified by adesso and the client at the workshop:

1. User story 201: recommendations for scientific documents

In this user story, the system should recommend related documents to users when they run a search so that they can provide a link to them directly in their articles. The maximum number of recommendations is predefined, with the output results shown in a table. The pipeline rhythm – whether event-driven or as a batch process – is preset. The acceptance criteria include the successful presentation of the recommendations for the relevant article.

2. User story 202: evaluation of recommendations

The aim is to give researchers the opportunity to evaluate the documents offered to them. The feedback is defined and stored in a separate database. The acceptance criteria include a simple feedback function. It is important to note that the aim of the feedback function is to draw conclusions about improvements to the prompt or embedding, without automatically making improvements to the search.

3. User story 203: notifications when new documents are available

A function is implemented for authors that notifies them when new documents have been added to the database that might be relevant to existing result types. To use this function, the document must be available in the database, the pipeline rhythm must be defined (either event-driven or as a batch process) and a defined notification process and format (in the form of a table containing new recommendations) must be in place. The acceptance criteria work on the assumption that the author will be notified when new suitable studies are available.

The workshop in five steps

One of the key milestones in the project was the four-hour workshop held with utopia, which focussed on EPIC 2. The aim of the workshop was to jointly identify and specify use cases in order to develop a UI prototype on this basis. The workshop included the following steps:

1. Status quo – verifying the pains and gains of the Zotero UI:

Pains (weaknesses) and gains (strengths) are analysed in order to respond in a targeted manner to the needs of users. In addition, the pains are identified to make targeted improvements possible, while the gains serve as the basis for further growth.

The challenges and successes of the existing Zotero user interface were analysed in one of the first steps and compared with the findings from the pain point canvas. This information was then used to identify current pains and gains.

2. Identifying the users’ needs and analysis path:

Personas are an easy way to really put yourself in the shoes of the users. They help in understanding the diversity of the target group and ensure that the development process focuses on the needs of users.

Defining a persona (in this case an editor) made it possible to understand the needs and the analysis path of the users. Personas are user models that characterise the behaviour, goals, expectations and motivations of a target group.

3. Preparing the use cases:

The persona served as the starting point when it came time to preparing detailed use cases. The pain relievers and gain creators were derived based on the value proposition canvas in order to better address the needs of users.

Specifying the use cases involved the following steps:

• 3.1 Writing articles and/or creating research/slides for customer workshop (consulting):Editors can use the system to write a new article and search for suitable publications. This may involve one or more publications.
• 3.2 Updating articles and/or slides/research results: This function allows you to revise existing articles or update workshop material and research results. Both editors and consultants can be actively involved in this step.
• 3.3 Evaluating a passage of text and publication: Users can rate the texts and publications found in the AI application as suitable or relevant. The aim here is to continuously improve the system.
• 3.4 Receiving a notification: The system provides information about existing articles and notifies you when new suitable publications are available.

The underlying theory of why use cases are important: Use cases define the interactions between the user and the system. They ensure that the functions offered by the system meet the actual needs of the users and provide a clear structure for development.

4. Prototyping:

Prototyping makes it possible to get an idea of what the planned solution will look like before it is fully implemented. This step is helpful in identifying potential problems early on and aided communication between developers and users.

The requirements for the tool were summarised and prioritised in the feature canvas, which served as a central repository for the specification of the use cases and revealed where there were issues that disrupted the users’ workflow.

Results from the feature canvas workshop

As part of the ‘Intelligent knowledge database – support during Aleph Alpha integration’ project, the feature canvas workshop was used to define clear priorities and requirements for the tool development process. The requirements identified can be divided into three categories.

• Must-haves: One of the main focuses was on functions that are essential for the efficient operation of the tool. This included a function to record the title, author, year of publication and the institution that is the source of the article. The grouping of publication series (threads) and a quick overview of study results and designs were other must-haves. Core requirements were the relevance of the publications, a clear order for the search results and the option to filter by publication category. In addition, it should also be possible to highlight specific passages of text in the sources, while the ability to search for publications by keyword and the option to filter and sort by date were other key criteria. The system should also show the relevant locations in the publications and offer a feedback tool in the form of a traffic light system.
• Should-haves: The second category includes functions that are not absolutely necessary but offer considerable added value. This could be notifications sent by e-mail, for example, to notify users of relevant updates. In addition, it should also be possible to follow specific topics in order to stay up to date. The ability to identify more recent publications in the same series and reference to newly available studies were other desired functions. Finally, it should also be possible to filter and sort by methodology and study design.
• Good-to-haves:The third category includes functions that offer added convenience but are not considered essential. This could be flexible modes to meet different needs of users and the option to adjust the relevance score using sliders. A citation generator (fact box) could also make it easier to use and process the content. In addition, continuous querying and feedback from users on the trustworthiness of the publications would be great to have as well.
  The clear prioritisation of these requirements provides a structured roadmap for the development phase, with the focus on the must-haves, followed by the should-haves and finally the good-to-haves. This approach ensures that the tool meets the basic requirements at the start and can later be expanded to include additional functions to ensure a higher degree of customisation and user-friendliness.

5. Feedback:

User feedback makes it possible to constantly improve the system and adapt it to meet changing user needs. Iterative feedback helps make the system more user-friendly and increases acceptance. The feedback received on the prototype was very positive. Users really liked the new functions and were happy with how easy the system is to use.

The defined use cases and features provide a clear framework for implementation of functions that will make editorial work easier over the long term. The results from the feature canvas workshop laid the foundation for the development of a user-friendly, feature-rich tool that increases editorial efficiency and revolutionises the use of knowledge databases.

The project showcases advances made in technology and highlights the added value generated when AI experts, software developers and the actual users of the system work together. The further development and implementation of the defined functionalities promise to deliver an innovative solution to meet the challenges faced when performing editorial work in the digital world.

Would you like to learn more about the solutions and services adesso offers in and around AI? Then take a look at our AI website or check out our other blog posts on the subject of AI.

Would you like to find out more about exciting topics from the world of adesso? Then take a look at our previous blog posts.