How to Set Up Role-Based Access for AI Memory
Why AI Memory Needs Access Control
Without access control, every user and AI agent in the organization can retrieve every memory. This creates three problems. First, sensitive information leaks through AI-generated responses. An AI assistant that has access to salary data, HR investigations, or M&A plans will surface that information when a query is semantically similar, regardless of whether the user should see it. Second, retrieval quality degrades as the memory pool grows. When the system searches across every memory in the organization, irrelevant results from other teams dilute the relevant results from the user's own context. Third, regulatory violations occur when personal data or classified information is accessible to users who should not have it, creating liability under GDPR, HIPAA, and similar frameworks.
RBAC is the most practical access control model for enterprise memory because it maps directly to organizational structure. Most organizations already have role definitions for their other systems (identity providers, SaaS tools, internal applications). Extending these roles to cover memory access is a natural step that avoids creating a separate access management system.
Step-by-Step Implementation
Start by listing the roles that exist in your organization and how they map to information access needs. Common role patterns: individual contributor (accesses their own and their team's memories), team lead (accesses team memories plus cross-team coordination memories), department head (accesses all memories within the department), executive (accesses organization-wide strategic memories), admin (manages access policies but does not necessarily see all memory content), and compliance officer (accesses audit trails and deletion records). Pull roles from your identity provider (Okta, Azure AD, Google Workspace) rather than maintaining a separate role list.
Every memory is assigned a visibility level when it is created. Five levels cover most organizations: personal (visible only to the creator), team (visible to the creator's team), department (visible to everyone in the department), organization (visible to all employees), and restricted (visible only to explicitly named roles or individuals). The default visibility should be the most restrictive level that makes sense for your culture. Most organizations default to team visibility and allow contributors to escalate to department or organization when the knowledge is broadly relevant.
{
"memory": {
"content": "Architecture decision: moving checkout from monolith to microservice",
"visibility": "department",
"namespace": "team:checkout",
"owner": "user:jane.smith",
"created_at": "2026-05-12T10:30:00Z",
"tags": ["architecture", "decision"]
}
}The permission matrix maps each role to the memory visibility levels and namespaces it can access. Express this as a table where rows are roles and columns are visibility levels. Each cell specifies read access, write access, or no access. The matrix also specifies which namespaces each role can access. A team member role might have read/write access to their team namespace, read access to their department namespace, and read access to the organization namespace, but no access to other teams' namespaces unless explicitly granted.
PERMISSION_MATRIX = {
"individual_contributor": {
"personal": {"read": True, "write": True},
"team": {"read": True, "write": True},
"department": {"read": True, "write": False},
"organization": {"read": True, "write": False},
"restricted": {"read": False, "write": False}
},
"team_lead": {
"personal": {"read": True, "write": True},
"team": {"read": True, "write": True},
"department": {"read": True, "write": True},
"organization": {"read": True, "write": False},
"restricted": {"read": False, "write": False}
},
"department_head": {
"personal": {"read": True, "write": True},
"team": {"read": True, "write": True},
"department": {"read": True, "write": True},
"organization": {"read": True, "write": True},
"restricted": {"read": True, "write": False}
}
}The critical architectural decision is where to enforce access control. Enforcing at query time means the memory system evaluates the requesting user's roles, determines which visibility levels and namespaces they can access, and filters the retrieval results to include only permitted memories. This is better than enforcing at storage time (tagging memories as "for role X") because roles change. When a person is promoted from individual contributor to team lead, their next query should immediately reflect their new access level without re-tagging any memories.
def query_memories(user_id, query_text, namespace=None):
user_roles = get_user_roles(user_id)
accessible_levels = get_accessible_visibility_levels(user_roles)
accessible_namespaces = get_accessible_namespaces(user_roles)
if namespace and namespace not in accessible_namespaces:
raise PermissionError(f"No access to namespace {namespace}")
results = vector_search(
query_text,
namespaces=accessible_namespaces if not namespace else [namespace]
)
filtered = [
r for r in results
if r.visibility in accessible_levels
or r.owner == user_id
]
log_access_event(user_id, query_text, len(filtered))
return filteredWhen a user's role changes (promotion, team transfer, departure), their memory access must change immediately. This means role assignments cannot be cached for long periods. Use your identity provider as the source of truth for roles, querying it on each memory access or caching roles with a short TTL (5 to 15 minutes). When someone leaves the organization, their personal memories should be handled according to your offboarding policy: transferred to their manager, archived, or deleted. Their contributions to team and department namespaces should persist because that knowledge belongs to the organization.
Write test cases that verify access control for each role. The test for an individual contributor should confirm they see team memories, see department-wide memories (read only), do not see other teams' memories, and do not see restricted memories. The test for a team lead should confirm they see everything the IC sees plus they can write to department namespaces. Include edge cases: what happens when a user has two roles (union or intersection of permissions), what happens when a memory's visibility is changed after creation (existing access events in the audit trail are preserved), and what happens when a namespace is deleted (memories become inaccessible but are not deleted until explicitly purged).
Integration with Identity Providers
Rather than managing roles within the memory system, integrate with your existing identity provider. Map identity provider groups to memory system roles using SCIM provisioning or SAML attributes. This ensures that role changes in your HR system propagate to memory access automatically, without manual synchronization. Most organizations already manage group memberships in Azure AD, Okta, or Google Workspace, so the role data is available, it just needs to be consumed by the memory system's permission engine.
Adaptive Recall integrates with standard identity providers through SAML and SCIM. Roles defined in your identity provider map directly to memory visibility levels, and role changes propagate in real time. The permission engine evaluates access on every query, and the audit trail records which role was active when each access occurred.
Enterprise access control without the engineering overhead. Adaptive Recall enforces role-based memory access that integrates with your existing identity provider.
Get Started Free