At the recent Codegeist Hackathon hosted by Atlassian, we decided to build an app for Confluence that leverages AI and Atlassian’s Forge APIs to help teams bridge gaps in internal knowledge bases. Read on to learn about what we built, why we built it, and what we learned along the way.
Check out our submission page for more information!
We built Magic Links, a context menu app designed specifically for Atlassian Confluence users.
Here’s how the app works for Confluence users:
- Text selection: Users highlight a section of their text that they want to find strong links for.
- AI link recommendations: Upon activating the Magic Links app, we use AI (ChatGPT) to find relevant Confluence pages that are strong matches for one or more of the selected text’s words to link to. To help the AI, we use Forge’s APIs to provide it context from Confluence (Page titles and the first 100 characters of each page). The AI is designed to provide a confidence score for each suggested link.
- Review link suggestions: The user can review all the suggested links provided by ChatGPT and accept the ones they find the most relevant.
- Insert and enhance: By clicking the “Insert Links” button, the selected text is updated — again, with Confluence’s APIs — with the relevant links. Now this Confluence page is better connected with other pages in the space.
The digital landscape of knowledge is vast, yet often, valuable insights remain isolated due to a lack of interlinking in expansive wikis like Confluence.
This means team members might miss out on critical context from other places in the wiki and would need to spend time manually finding pages to link to for the pages they create.
Magic Links is our attempt at addressing this, turning isolated islands of information into a connected archipelago of insights. Our overall vision is to use AI to deeply connect internal knowledge graphs to streamline team efficiency and collaboration.
We initially embarked on a journey to leverage word embeddings for creating links.
Embeddings, in the realm of machine learning, convert words or phrases into vectors of real numbers. The beauty lies in their ability to capture the semantic essence of a word (or group of words), making them potentially game-changing for our problem: correlating text with relevant wiki pages.
Here’s how we executed this approach for the hackathon:
- Forge’s API: First, we used Atlassian’s Forge APIs to capture the title and first 100 characters of every page within the space.
- OpenAI embeddings: Then we used OpenAI’s embedding model to embed each of those texts into a single numerical vector.
- More embeddings: We also embedded each of the non-common words (and phrases) within the text selected by the user on the Confluence page.
- Embedding comparison: We compared the embedding of each word from step 3 with all the embeddings of Confluence pages. Using cosine similarity, we measured the distance between these embeddings and returned back matches that were the “closest” together
However, we found that the quality of the suggested link matching wasn’t up to par. We believe that the reason for this is that when we embedded a single word (or phrase) from the user’s selected text, we didn’t provide it any other context so it couldn’t suss out the semantic meaning of that word in the context of our Confluence space. This meant simply comparing the word embeddings with that of a batch of text (a page’s title and first 100 characters) came back with link matches that weren’t quite helpful.
Pivoting from our initial approach, we decided to harness the prowess of ChatGPT, tasking it with the heavy lifting of identifying pertinent links.
Here’s how we executed this approach:
- Forge’s API: Just like before, we used Atlassian’s Forge APIs to capture the title and first 100 characters of every page within the space.
- Engineer the initial OpenAI prompt: We took great care, and a lot of trial-and-error, to engineer the best prompt to tell ChatGPT exactly what we wanted it to do. This took a significant amount of iterations but it was critical getting the right results.
- Add in page information: Within the prompt, we took all the page information from step 1 (title and first 100 characters) and added each one to the prompt. Then we gave the entire prompt to ChatGPT to try and suss out what it needed to do.
By feeding ChatGPT a meticulously crafted prompt coupled with the selected text and wiki page details, it adeptly discerned strong link candidates. This approach delivered on our vision for Magic Links but it faces obvious scaling issues when dealing with Confluence spaces with hundreds of pages. We touch on this in the next section.
Atlassian’s Forge unfurled an arena of possibilities, proving to be both powerful and conducive for rapid prototyping during the hackathon. As we move forward, we’re brimming with ideas:
- Connecting more Atlassian products: We’re keen on integrating more Atlassian products like JIRA and Bitbucket into the mix, elevating Atlassian’s interconnected experience by providing relevant links to those surfaces.
- Scale our calls to ChatGPT: We’re excited to use Forge’s async events API to run parallel calls to ChatGPT with a portion of the pages in order to handle arbitrarily large Confluence spaces.
- Embedded links: Our ambition is to merge the strengths of both our embedding and ChatGPT approaches, aiming for unparalleled link suggestion accuracy.
Our journey with Atlassian’s Forge APIs has been nothing short of exhilarating. As we stand on the cusp of this new frontier, we eagerly await to see the transformative impact Magic Links will usher for teams worldwide.