AI Framework Group

Vision

The AI Framework group is focused on how to support other product groups at GitLab with the AI Abstraction Layer, and GitLab AI feature development.

👌 Team OKRs

If you’re interested in the team’s Objectives and Key Results (OKRs), you can find them on GitLab.

🚀 Team Members

Engineering Manager & Engineers

Name	Role
David O'Regan	Engineering Manager, AI Framework
Alexander Chueshev	Staff Machine Learning Engineer, AI-powered:AI Framework
Alejandro Rodríguez	Senior Backend Engineer, AI Powered:AI Framework
Andrew Fontaine	Staff Frontend Engineer, AI-powered:AI Framework
Bruno Cardoso	Senior ML Engineer, AI-powered:AI Framework
Denys Mishunov	Staff Frontend Engineer, AI-powered:AI Framework
Frédéric Caplette	Staff Frontend Engineer, AI-powered:AI Framework
Halil Coban	Senior Backend Engineer, AI-powered:AI Framework
Backend Engineer	Backend Engineer, AI-powered:AI Framework
Lindsey Shelton	Frontend Engineer, AI-powered:AI Framework
Małgorzata Ksionek	Staff Backend Engineer, AI Powered:AI Framework
Michael Thomas	Backend Engineer, AI-powered:AI Framework
Nathan Weinshenker	Intermediate Backend Engineer", AI Powered:AI Framework
Shinya Maeda	Staff Backend Engineer, AI-powered:AI Framework
Surabhi Suman	Senior Backend Engineer, AI-powered:AI Framework

Product, Design & Quality

Name	Role
Katie Macoy	Senior Product Designer, AI-powered:AI Framework
Senior Product Designer	Senior Product Designer, AI-powered:AI Framework
Ottilia Westerlund	Security Engineer, Fulfillment (Fulfillment Platform, Subscription Management), Govern (Security Policies, Threat Insights), Monitor (Observability), Plan (Product Planning), AI-powered (Duo Chat, AI Framework, AI model validation, Custom models)
Pini Wietchner	Senior Product Manager, AI-powered:AI Framework
Suzanne Selhorn	Staff Technical Writer, AI-powered (Duo Chat, AI Framework, AI model validation, Custom models)

Team members who are supporting this team on a short-term basis are located here where “Work priorities” include “AI Framework.”

☕ Team Responsibilities

Team responsibilities include

• Facilitating the integration of AI capabilities throughout GitLab by leveraging the AI Abstraction Layer and AI Gateway.
• Guaranteeing the presence of comprehensive global observability, monitoring, documentation, and enhancements in latency.
• Providing essential support for our AI chat system framework.
• Incorporating support for new AI providers when required.
• Assisting with the production support and ensuring the readiness of the AI Gateway.
• LLM inference support, including prompt enginering, response evaluation, fine tuning, evaluation, and more.

☎️ How to reach us

Depending on the context here are the most appropriate ways to reach out to the IDE Group:

• Slack Channel: #g_ai_framework
• Slack Groups: @ai-framework (entire team) and @ai-framework-engs (just engineers)

📦 Team Processes

📆 Regular team meetings

❗️Important: For every meeting, the AI Framework team’s meeting document should be used, and filled with the meeting notes, as well as references to any other sync meeting agendas/notes/recordings which have recently occurred. This will make it easier for people to find any meeting notes.

Team Meetings

1. Weekly Refinement/Weight Assignment Meeting

   • When: Every Monday, alternating between 09:00 AM GMT+1 and 17:00 PM GMT+1
   • What: This meeting replaces the previous Work Assignment Meeting and focuses on refining and assigning weights to issues. The Engineering Manager and Product Manager ensure all issues are properly refined and weighted.

2. Duo Workflow Weekly

   • When: Every Monday at 17:00 – 18:00 PM GMT+1 Wednesday at 09:00 – 10:00 GMT+1.
   • What: This meeting focuses on reviewing Duo Workflow related items.

3. Think BIG sessions

   • When: Once a month
   • What: This session will alternate between EMEA/AMER and APAC regions every other month, focusing on broad discussions about group dynamics and strategic planning.

Shared calendars

• AI Framework PTO (Calendar ID: c_eca9440729dba2cbd88b3117fa70839836fb5811cb072132b94c52f912a31bf5@group.calendar.google.com)
• AI-Powered Stage Calendar (Calendar ID: c_n5pdr2i2i5bjhs8aopahcjtn84@group.calendar.google.com)

AI Framework team members should sync your PTO events with AI Framework PTO calendar.

🖖 Weekly EM Updates

Each week the team EM provides a Weekly Status update issue which aims to capture the most important items for the team to be aware of. These can be found here.

📚 AI Framework Board Outline

Our workflow process for our board is outlined below.

1. Open 📝: This list includes all identified issues. An engineering manager will be assigned if either the Milestone or the “workflow::ready for development” label is missing.
2. workflow::design ✏️: During this phase, issues undergo design refinement. After design considerations are integrated, the “ready for development” label is applied.
3. workflow::refinement 🧪: Issues in this stage are undergoing engineering refinement to ensure the proposed solution meets all requirements. Once refined, the “ready for development” label is applied.
4. workflow::ready for development 🎯: Issues that are prioritized and assigned to a specific milestone are moved to this list, and the “ready for development” label is applied.
5. workflow::in dev 👩‍💻: When a developer begins work on an issue, they should move it to this list and apply the “in dev” label.
6. workflow::in review 👀: After development is complete, the issue moves to this list, and the “in review” label is applied.
7. workflow::verification ✅: Following a successful code and UX review, the issue should be moved to this list and the “verification” label should be applied.
8. workflow::complete 🎉: Once the issue is verified and confirmed to be working properly, it should be moved to this list, the “complete” label should be applied, and the issue should be closed.

📝 Issue Priority

To ensure that our developers are aware of the priority of their work, we use three labels:

• AIF-Priority::1: This label is for issues that are of the highest priority. These issues should be addressed first.
• AIF-Priority::2: This label is for issues that are of medium priority. These issues should be addressed after all Priority 1 issues have been resolved.
• AIF-Priority::3: This label is for issues that are of lower priority. These issues should be addressed after all Priority 1 and Priority 2 issues have been resolved.

🔄 Processes

🗓️ Weekly Process

1. Open 🆕: (DRI: PM) Capture and list new issues as they arrive. This stage serves as the initial holding area for issues awaiting classification and prioritization.
2. Workflow Design ✍️: (DRI: Designer) Move issues into this lane for design refinement, focusing on user experience and interface design.
3. Workflow Refinement 🔧: (DRI: Engineering Lead) Conduct technical refinement to ensure feasibility and adherence to standards.
4. Workflow Ready for Development 🎯: (DRI: PM/EM) Transition fully refined and specced issues here, ready for developers to start coding.
5. Workflow in Development 💻: (DRI: Assigned Developer) Implement and build the specified features and functionalities.
6. Workflow in Review 🔍: (DRI: QA/Reviewer) Perform thorough reviews and testing, including code reviews and usability testing.

📝 Issue Guidelines

These guidelines apply to all issues we use for planning and scheduling work within our group. Our Engineers can define specific implementation issues when needed, but the overall goal for our issues are as follows:

• Provide a meaningful title that describes a deliverable result.
  • ✅ Add the new GitLab Duo chat package as a Vue2 extension
  • ✅ Chat: move away from using OpenAI embeddings
  • ❌ Make Chat better
• Provide a meaningful description that clearly explains the goal of the issue, and provide some technical details if necessary.
• Should there be critical implementation steps or other useful ways to create small tasks as part of the issue, please use a checklist as part of the issue descriptions.
• The issue should have a weight, milestone and workflow label assigned.
• Design and frontend engineering use one issue. The same issues moves from workflow::design to workflow::refinement to workflow::ready for development. This ensures that there is a single source of truth for customer-facing issues. If a design issue is too large to be implemented, it may be promoted to an epic.

It’s okay to create specific engineering-driven implementation issues for more complex features. These would be called Child Issues and they should always link back to their parent. If one issue would spawn many child issues, consider creating an Epic.

🏋 Weighting and Estimation Process

To assign weights to issues effectively, it’s important to remember that issue weight should not be tied to time. Instead, it should be a purely abstract measure of the issue’s significance. The team uses the Fibonacci sequence starting from weight 0:

• Weight 0: Reserved for the smallest and easiest issues, such as typos or minor formatting changes, or very minor code changes with no tests required.
• Weight 1: For simple issues with little or no uncertainty, risk or complexity. These issues may have labels like “good for new contributors” or “Hackathon - Candidate”. Example: Changing copy text which may be simple but take some time.
• Weight 2: For more involved issues which are still straightforward without much risk or complexity, but may involve touching multiple areas of the code, and updating multiple tests.
• Weight 3: For larger issues which may have some unforeseen complexity or risk, or require more extensive changes, but is still not large enough to warrant breaking down into smaller separate issues.
• Weight 5: Normally, this weight should be avoided, and indicates that the issue ideally should be broken down into smaller separate issues. However, in some cases, a weight of 5 might still be prioritized. For example, if there is a large amount of manual updates to be made which will require a large amount of effort, but doesn’t necessarily involve significant risk or uncertainty.
• Weight 8/13+: Weights above 5 are used to clearly indicate work that is not yet ready to be assigned for implementation, and must be broken down because it is too large in scope to start implementing, and/or still has too many unknowns/risks. This weight is temporarily assigned to “placeholder” issues to capture the scope of the effort in our velocity-based capacity planning calculations.

📝 AI Feature Evaluations Guidelines - Evaluate like I am 5

See the Evaluate like I am 5 Project and read the docs here.

👏 Communication

The AI Framework Team communicates based on the following guidelines:

1. Always prefer async communication over sync meetings.
2. Don’t shy away from arranging a sync call when async is proving inefficient, however always record it to share with team members.
3. By default communicate in the open.
4. All work-related communication in Slack happens in the #g_ai_framework channel.

⏲ Time Off

Team members should add any planned time off in the “Time Off by Deel” slack app, so that the Engineering Manager can use the proper number of days off during capacity planning.

🤙 Ad-hoc sync calls

We operate using async communication by default. There are times when a sync discussion can be beneficial and we encourage team members to schedule sync calls with the required team members as needed.

🔗 Other Useful Links

🌍 AIGW Region Deployments

• 🚀 Staging
• 🌐 Production

📝 Dashboards (internal only)

• Requests per provider
• Error budgets
• AI Gateway SLIs
• GCP quota limits
• LLM Sidekiq completions
• Duo Chat Question Categorization
• Security Issues
• Reliability Issues
• Sentry via CompletionWorker
• Sentry via Feature Category
• Monthly Retros
• Chat QA Evaluation
• Chat REST API Error Ratio

📹 GitLab Unfiltered Playlist

The AI Framework Group collates all video recordings related to the group and its team members in a playlist in the GitLab Unfiltered YouTube channel.

View page source - Edit this page - please contribute.