💼 Hiring Quest - Senior AI Engineer (Generative AI & Azure AI Platform) @ Kentrick.ai

👋We are Kentrick.ai, a fast-growing startup building production-grade AI solutions and platforms.

We are looking for a Senior AI Engineer - Generative AI & Azure AI Platform who can take AI systems beyond prototypes and build reliable, secure, observable, and cost-efficient production solutions.

This is a hands-on senior position for an engineer who can make strong technical decisions, own architecture end-to-end, and clearly explain the trade-offs behind those decisions

🕘 Start Date: Immediate

📍 Location: Cairo

⏰ Employment: Full-Time

💰 Compensation: EGP 120,000–150,000 egp gross per month

🎖️ Experience: 8-10+ years overall AI / ML / Data Engineering experience, including strong hands-on production GenAI experience

Build Environment: Local and no-cost; Azure architecture is design only

⚒️How the Hiring Quest Works

1. Register for the quest

2. Receive the challenge and assessment pack

3. Complete the required scope at any time before the submission deadline

4. Submit your complete solution and short decision-walkthrough video before the deadline

5. Top candidates join a 60-90 minute technical review and defend key decisions

6. One candidate will be selected for the position

Your invitation will state the submission deadline, time zone, and delivery channel. Complete the required scope before that deadline and confirm that reviewers can access both the repository and video.

🔎Who We're Looking For

1. We are looking for someone with strong production experience - not only experience experimenting with LLM APIs.

2. You should be comfortable with:

3. Azure AI Foundry, Azure OpenAI, Azure Machine Learning, and Azure AI Search

4. Production RAG architectures and evidence-grounded generation

5. Vector, keyword, hybrid, and semantic retrieval

6. LLM evaluation, observability, and regression analysis

7. Prompt engineering and orchestration frameworks

8. Open-source foundation models and modern model-serving approaches

9. MLOps, LLMOps, containers, APIs, and CI/CD practices

10. Responsible AI, indirect prompt injection, and LLM security

11. Entra ID, managed identities, Key Vault, private networking, and secure Azure architectures

12. Expert-level Python and strong software engineering practices

13. Clear technical communication, prioritization, and end-to-end ownership

Beyond implementation, you should be able to balance:

Model Quality x Retrieval Quality x Latency x Cost x Security x Reliability x Maintainability

🧠The Hiring Challenge

Enterprise Knowledge Intelligence Platform

ApexOne Group operates across HR, Legal, Finance, Engineering, Security, Operations, and Sales. Employees rely on policies, contracts, SOPs, technical documentation, knowledge articles, and structured tables. The information is duplicated, permission-sensitive, and updated frequently.

A prototype AI assistant reduced search time, but it also produced obsolete guidance, irrelevant evidence, confidential-data exposure, unsupported claims, and unsafe responses to malicious document content. Kentrick.ai has been asked to redesign the core before wider rollout.

Your task is to build the focused local core of a production-ready Enterprise Knowledge Intelligence Platform and then design how it should evolve on Azure. This should not be treated as a simple chat-with-PDF application.

The quest has three clear parts

Part 1: Build and evaluate a locally runnable RAG prototype using the supplied assessment pack.

Part 2: Design the secure, scalable Azure production architecture. This part is design only.

Part 3: Record a short walkthrough of the decisions that shaped both solutions.

Core business question

"What is our process for approving a new enterprise vendor?"

REQUIRED BUSINESS OUTCOME  Give a useful, evidence-backed answer when trustworthy information is available and a safe, honest response when it is not.

Production assumptions

1. Knowledge estate: 60,000 documents / about 180 GB.

2. Change rate: Approximately 200 creates, updates, retirements, or deletions each business day.

3. Usage: 5,000 employees; peak 20 requests/second.

4. Experience: p95 response time below 6 seconds.

5. Security: Zero unauthorized disclosure or influence from inaccessible information.

6. Residency: Services and data stay within an approved Azure region.

7. Reliability: High-risk requests fail safely during retrieval or model outages.

These assumptions are for the Azure production design. The local prototype is evaluated only against the supplied corpus, identities, and four cases; it need not reproduce production scale, update volume, availability, or latency. Security applies to both parts.

Assessment pack

Here you can download ( Link ) a small common corpus containing:

1. Company-wide internal and restricted documents from several departments

2. Current, retired, and apparently conflicting guidance

3. Narrative text and a table-based document

4. A document containing malicious instructions

5. Sample user identities and role/department entitlements

6. A normalized JSONL version of the corpus with content and metadata ready for local retrieval

🛠️Part 1 - Build the Local RAG Prototype

Design, implement, and evaluate a locally runnable retrieval-grounded knowledge assistant using the supplied normalized corpus. The original PDF and DOCX files are provided as business evidence and reference material; parsing them is not part of the required implementation.

The assessed run path must work on a standard CPU-only workstation without a GPU, Azure account, paid credits, or cloud service. Model choice - including whether answer generation is needed - is an engineering decision you must justify. Document any installation downloads and the exact commands required to run the solution locally.

Required outcomes

1. A simple runnable interface that receives a question and one of the supplied user identities

2. Useful answers grounded in information the user is permitted to access, with traceable source evidence

3. Safe, useful behaviour when the available evidence is missing, outdated, conflicting, restricted, or untrustworthy

4. No influence from information the user is not permitted to access, including answers, references, diagnostics, and other observable output

5. Document content cannot change company rules, expand permissions, disclose protected information, or initiate unauthorized action

6. Four repeatable end-to-end cases showing how the business incidents are handled

7. Reasonable behaviour for equivalent question wording and input variations; prewritten answers for the mandatory cases are not acceptable

Local design explanation

Include one compact diagram or README flow showing how a question becomes an answer or safe response. Identify the important trust boundary and explain the decisions that protect access, evidence quality, and source traceability. Include the main alternatives you considered and the limitations that remain.

🏗️Business Incident 1 - The Wrong Policy Became the Answer

A procurement manager asked how to approve a new enterprise vendor. The assistant returned a retired process because it was semantically similar to the question. The corpus contains several versions, a table that updates approval thresholds, and a Legal memo that narrowly qualifies part of the current policy.

REQUIRED BUSINESS OUTCOME  The answer must be supported by relevant, traceable evidence and must not present obsolete or materially conflicted guidance as authoritative.

Evidence to include

1. One grounded answer with enough source evidence for a reviewer to verify the important claims

2. One case showing what happens when current, retired, or apparently conflicting information is available

3. A short explanation of how the solution decides which evidence can be trusted for the answer

🏹Business Incident 2 - The Convincing Unsupported Answer

A manager asked for a supplier's contractual response time. The retrieved agreement mentioned support but contained no SLA. The prototype invented a plausible target and cited evidence that did not support the claim.

REQUIRED BUSINESS OUTCOME  Every important claim must be supported. When evidence is missing or weak, the system should refuse or qualify the answer and state what evidence is needed.

Include one unanswerable contractual-SLA case with a clear expected response.

🚨Business Incident 3 - Security Failure

An Engineering employee searching for company leave guidance received details from a confidential HR investigation. Another indexed document tells the assistant to ignore company rules, reveal confidential information and secrets, and invoke available tools.

REQUIRED BUSINESS OUTCOME  Employees must receive only information they are allowed to use. Restricted material must not affect unauthorized outcomes or operational records. Instructions hidden inside documents must not change company rules, expand access, reveal information, or trigger unauthorized actions.

Evidence to include

1. The same or equivalent request from the Engineering employee and the authorized HR employee

2. Evidence that each employee receives an outcome consistent with the supplied entitlements

3. The observed outcome when the system encounters the malicious document

4. The protections you chose, the business risk each addresses, and the limitations that remain

Any restricted case detail affecting an unauthorized answer, reference, diagnostic, or operational record is a critical security failure.

⚓Business Incident 4 - Undetected Regression

A prompt or retrieval change made answers more fluent but caused one of the critical business safeguards to fail. The team discovered the regression only after release.

REQUIRED BUSINESS OUTCOME  The local prototype must make the four critical behaviours repeatable and release-blocking. Production diagnostics, latency, and cost observability should be addressed in the Azure design rather than implemented locally.

Evaluation evidence

1. A small version-controlled dataset containing the four required cases and one repeatable evaluation command

2. Results showing whether the four business incidents are handled safely

3. A short statement of which failures would block a release and why

Mandatory case families

1. Policy authority: current guidance wins over retired or materially conflicting evidence

2. Unsupported SLA: missing contractual evidence is communicated without invention

3. Authorization: authorized and unauthorized supplied users receive permission-consistent outcomes

4. Malicious document: embedded instructions cannot override company policy or expand access

   ---

💎Part 2 - Design the Azure Production Architecture

DESIGN ONLY. Do not create an Azure account, purchase credits, deploy resources, or provide screenshots of a running Azure environment.

Design how the local prototype would evolve into a secure, scalable, observable, and cost-conscious Microsoft Azure RAG platform for the production assumptions in this quest. The proposed cloud architecture must use Azure services only; do not propose AWS, GCP, or another cloud provider.

Submit and explain

1. One Azure architecture diagram showing the main user and content flows and the important trust boundaries

2. A component map covering how knowledge is stored and changed, how employee questions are answered, how access is enforced, and how the platform is operated and released

3. For every important component or service: the specific Azure service, its responsibility, why you selected it, and one meaningful Azure alternative or architectural trade-off

4. How the design prevents restricted information from affecting unauthorized processing or diagnostics

5. How the design meets the stated production assumptions when content changes, work is retried, dependencies fail, usage grows, or quality, latency, and cost regress

6. A practical migration path from the local prototype and the first two production investments you would prioritize

Generic cloud labels are not sufficient. Name the relevant Azure services and connect them to data flow, security reasoning, operating concerns, and trade-offs.

Not required locally: original-file parsing, a user interface, a separate authentication service, cloud deployment, or production-scale load and operations. Additional capabilities receive no extra credit unless they materially improve a required business outcome. The Azure design still covers production content lifecycle, identity, operations, scaling, reliability, cost, and deployment.

🎓Deliverables

1. Repository: Working focused local RAG source, four repeatable evaluation cases, clear structure, and no real credentials or confidential material.

2. README: Exact setup, run, and evaluation commands; compact local flow and design decisions; configuration; examples; assumptions; and known limitations.

3. Evaluation: Version-controlled cases, repeatable command, latest results, failures, conclusions, and proposed release blockers.

4. Azure design: Azure-only architecture diagram, named service explanations, security and operations reasoning, trade-offs, and migration priorities. Design only; no account or deployment required.

5. Video: A 5-7 minute decision-walkthrough link in the README, focused on the business incidents and your engineering decisions.

🧬Technical Review

Selected candidates will join a 60-90 minute technical review. You may be asked to explain or modify part of the local solution, debug an unseen business variation, and defend the Azure architecture and service choices.

The review is part of the assessment: you should understand and be able to modify everything you submit.

📹Part 3 - Short Decision Walkthrough

Submit a 5-7 minute screen recording focused on the decisions behind the local implementation and Azure production design. This is not a code tour: do not explain every file, class, function, or Azure service.

Show two short outcomes

1. One useful answer supported by evidence

2. One case where the system protects the business by refusing, clarifying, handling conflicting evidence, enforcing permissions, or resisting the malicious document

Briefly answer

1. Which business risk most influenced your local design, and what decision did you make because of it?

2. What important assumption or failure mode remains in the local solution?

3. Using the Azure diagram, trace one employee request and one content change, explaining the most important service choices and trust boundary.

4. What would you build first when moving to Azure, and how would you detect a quality, security, latency, or cost regression?

Add a private or unlisted link to the README and confirm access. Camera and slides are optional. If recording is a barrier, request a live or written alternative. We score clarity and judgment, not production polish.

🌐AI Usage Policy

Use generative AI tools sparingly. They may assist your work, but the submitted design, reasoning, and implementation must remain yours and understandable. Add a brief AI assistance note to the README covering:

1. Which tools you used

2. Which files, sections, tests, diagrams, or other artifacts received substantial assistance

3. What the tool generated or suggested and the underlying logic you chose to keep

4. What you changed and how you reviewed, secured, and validated the result

Raw conversation transcripts and line-by-line explanations are not required. During the review, you may be asked to explain, debug, or modify any AI-assisted artifact without using the tool. Substantial undisclosed or unexplained generated work may result in rejection.

📊Evaluation Criteria

1-Retrieval and grounded behaviour - 25%

Relevant evidence, traceable claims, safe uncertainty, and practical debugging.

2-Security and Responsible AI - 25%

Authorization is a system guarantee and untrusted content remains constrained.

3-Evaluation and tests - 20%

Repeatable evidence, meaningful adversarial cases, and credible release-blocking checks.

4-Architecture and software quality - 20%

Focused implementation, clear boundaries, a credible Azure-only production design, and maintainable code.

5-Judgment and ownership - 10%

Good prioritization, honest limitations, defensible decisions, and deep understanding.

Critical concerns

1. Unauthorized information affects an answer, reference, diagnostic, or operational record

2. The system produces confident unsupported claims with no safe fallback

3. The submission cannot be run or tested using the documented instructions

4. The candidate cannot explain or safely modify substantial submitted work

5. Substantial AI-assisted work is undisclosed or cannot be explained and validated

   ---

💡What We Care About

A smaller system you understand deeply will score higher than a polished system you cannot explain, debug, or change.

The quest defines the outcomes and constraints, not a preferred implementation recipe. Selecting, testing, and defending an appropriate approach is part of the assessment.