Generative AI Consultant

Overview :

Our client will be hiring a Generative AI Testing and Evaluation Consultant focused on user adoption and production strategy for a short-term remote Contract.

This position reports to the Director of Data Science and will work with the IT Director. Expect to develop a timetable and workplan for the deliverables in the first week during the kickoff meeting.

The Technology for Development team collaborates with internal and external partners to unlock new possibilities to improve program quality, impact, and innovation, helping improve more lives through the power of technology and data. The mission and purpose are to drive sustainable program impact through responsible application of existing and emerging digital technologies. The IT division ensures that the global workforce has the infrastructure, security, support, and technology they need to fulfill their critical missions.

Purpose / Project Description:

This role will support IT by designing and implementing a user-testing protocol based on the current framework for Generative AI. The consultant will evaluate tools for the implementation and use of Generative AI at the enterprise level. Expect to focus on determining enterprise-level use cases for Generative AI, evaluate the utility of an existing custom-built platform (Digital Library Chatbot) against these use cases, and develop/execute a comparative testing format for at least two commercially available Generative AI solutions.

Responsibilities:

• Leverage previous work on Generative AI testing frameworks, define a clear testing process for enterprise AI tools
• Conduct guided and/or independent testing on the Digital Library Chatbot
• Recommend a procurement strategy for testing at least two commercially available enterprise Generative AI solutions (e.g. CoPilot, ChatGPT, Amazon Q)
• Conduct guided and/or independent testing on enterprise Generative AI Solutions
• Conduct a comparative analysis of enterprise solutions including financial comparisons

Deliverables:

The Consultant will provide as final deliverables:

• Report detailing the following elements:
  • Results of user testing for the Digital Library Chatbot and concise recommendations on enterprise-wide rollout
  • Results of user testing for commercially available enterprise Generative AI solutions and recommendations for procuring an enterprise-wide Generative AI solution

Qualifications:

• Strong experience in technology evaluation at the enterprise level, participatory research, and the use of Generative AI for aid/development
• 3 years of experience related to IT at aid/development organizations and/or Technology for Development

Preferred

• Experience with AI and its potential role in international development
• Experience with commercial off the shelf solutions for Generative AI
• Experience with software development and ICT at the enterprise level