Implement identity entities and groups

40min
|
Vault
Interactive

Deployment

Vault supports multiple authentication methods and also allows enabling the same type of authentication method on different mount paths. Each Vault client may have multiple accounts with various identity providers that are enabled on the Vault server.

Vault clients can be mapped as entities and their corresponding accounts with authentication providers can be mapped as aliases. In essence, each entity is made up of zero or more aliases. Identity secrets engine internally maintains the clients who are recognized by Vault.

Personas

The steps described in this tutorial are typically performed by operations persona.

Challenge

Bob has accounts in both Github and LDAP. Both Github and LDAP auth methods are enabled on the Vault server that he can authenticate using either one of his accounts. Although both accounts belong to Bob, there is no association between the two accounts to set some common properties.

Solution

Create an entity representing Bob, and associate aliases representing each of his accounts as the entity member. You can set additional policies and metadata on the entity level so that both accounts can inherit.

When Bob authenticates using either one of his accounts, the entity identifier will be tied to the authenticated token. When such tokens are put to use, their entity identifiers are audit logged, marking a trail of actions performed by specific users.

Note

If you are running Vault Enterprise or HCP Vault Dedicated, be sure to read the Why create entities? section at the end.

Prerequisites

To perform the tasks described in this tutorial, you need to have a Vault environment.

Refer to the Getting Started tutorial to install Vault
jq to process the JSON output for readability

Launch Terminal

This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.

Policy requirements

Unless you are running Vault for test or development purposes, such as dev mode (-dev), or with a short lived HCP Vault Dedicated dev tier cluster, it is recommended that you authenticate with Vault and acquire a tokens with an appropriate set of policies based on your role in the organization.

Policy requirements

If you are connecting to a non-development Vault server, your token must have a policy with following permissions:

# Configure auth methods
path "sys/auth" {
  capabilities = [ "read", "list" ]
}

# Configure auth methods
path "sys/auth/*" {
  capabilities = [ "create", "update", "read", "delete", "list", "sudo" ]
}

# Manage userpass auth methods
path "auth/userpass/*" {
  capabilities = [ "create", "read", "update", "delete" ]
}

# Manage github auth methods
path "auth/github/*" {
  capabilities = [ "create", "read", "update", "delete" ]
}

# Display the Policies tab in UI
path "sys/policies" {
  capabilities = [ "read", "list" ]
}

# Create and manage ACL policies from UI
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Create and manage policies
path "sys/policies/acl" {
  capabilities = [ "read", "list" ]
}

# Create and manage policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# List available secrets engines to retrieve accessor ID
path "sys/mounts" {
  capabilities = [ "read" ]
}

# Create and manage entities and groups
path "identity/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

If you are not familiar with policies, complete the policies tutorial.

Lab setup

In a terminal, start a Vault dev server with root as the root token.
```
$ vault server -dev -dev-root-token-id root
```
The Vault dev server defaults to running at 127.0.0.1:8200. The server is initialized and unsealed.
Insecure operation
Do not run a Vault dev server in production. This approach starts a Vault server with an in-memory database and runs in an insecure way.
Open a new terminal and export an environment variable for the vault CLI to address the Vault server.
```
$ export VAULT_ADDR=http://127.0.0.1:8200
```
Export an environment variable for the vault CLI to authenticate with the Vault server.
```
$ export VAULT_TOKEN=root
```

The Vault server is ready.

Create an Entity with Alias

You are going to create a new entity with base policy assigned. The entity defines two entity aliases with each has a different policy assigned.

Scenario: A user, Bob Smith at ACME Inc. happened to have two sets of credentials: bob and bsmith. Bob can authenticate with Vault using either one of their accounts. To manage the user's accounts and link them to identity Bob Smith in the QA team, you are going to create an entity for Bob.

For the simplicity of this tutorial, you are going to work with the userpass auth method. In reality, the user bob might be a username that exists in Active Directory, and bsmith might be Bob's username in GitHub. To mock the behavior, you are going to enable the userpass auth method at two separate paths: userpass-test and userpass-qa to mimic two different types of auth methods.

User lockout

As of Vault 1.13, the User lockout feature is enabled by default for the userpass, approle, and ldap auth methods.

Scenario Policies

path "secret/data/training_*" {
   capabilities = ["create", "read"]
}

path "secret/data/test" {
   capabilities = [ "create", "read", "update", "delete" ]
}

path "secret/data/team-qa" {
   capabilities = [ "create", "read", "update", "delete" ]
}

Note

This scenario assumes that the K/V v2 secrets engine is enabled at the secret path. If you are not sure, refer to the Versioned Key/Value Secrets Engine tutorial.

Create bob and bsmith users with appropriate policies attached.

Create base policy.

$ vault policy write base -<<EOF
path "secret/data/training_*" {
   capabilities = ["create", "read"]
}
EOF

Create test policy.

$ vault policy write test -<<EOF
path "secret/data/test" {
   capabilities = [ "create", "read", "update", "delete" ]
}
EOF

Create team-qa policy.

$ vault policy write team-qa -<<EOF
path "secret/data/team-qa" {
   capabilities = [ "create", "read", "update", "delete" ]
}
EOF

List all policies to verify that base, test and team-qa policies exist.
```
$ vault policy list
base
default
team-qa
test
root
```
Note
If you are using HCP Vault Dedicated, you will also see the hcp-root policy.

Enable the userpass auth method at userpass-test.

$ vault auth enable -path="userpass-test" userpass

Create a new user named bob in userpass-test where the password is training and test policy is attached.
```
$ vault write auth/userpass-test/users/bob password="training" policies="test"
```
Enable the userpass auth method at another paths, userpass-qa.
```
$ vault auth enable -path="userpass-qa" userpass
```
Create a new user named bsmith in userpass-qa where the password is training and team-qa policy is attached.
```
$ vault write auth/userpass-qa/users/bsmith password="training" policies="team-qa"
```

Execute the following command to discover the mount accessor for the userpass auth method.

$ vault auth list -detailed

Path              Plugin      Accessor                    ...
----              ----        --------                    ...
token/            token       auth_token_c5943123         ...
userpass-qa/      userpass    auth_userpass_8c7b8e0f      ...
userpass-test/    userpass    auth_userpass_264d4705      ...
...

Each userpass auth methods has a unique Accessor value to identify it.

Run the following command to store the userpass-test auth accessor value in a file named accessor_test.txt.
```
$ vault auth list -format=json | jq -r '.["userpass-test/"].accessor' > accessor_test.txt
```
The resulting file contains the accessor value (auth_userpass_XXXXX).
Similarly, run the following command to store the userpass-qa auth accessor value in a file named accessor_qa.txt.
```
$ vault auth list -format=json | jq -r '.["userpass-qa/"].accessor' > accessor_qa.txt
```

Create an entity for bob-smith, and store the returned entity ID in a file named, entity_id.txt.

$ vault write -format=json identity/entity name="bob-smith" policies="base" \
     metadata=organization="ACME Inc." \
     metadata=team="QA" \
     | jq -r ".data.id" > entity_id.txt

The resulting file contains the entity ID for bob-smith (e.g. 24204b50-22a6-61f5-bd4b-803f1a4e4726).

Now, add the user bob to the bob-smith entity by creating an entity alias. Set a custom metadata on the bob entity alias named, "account" and set its value to be "Tester Account".

$ vault write identity/entity-alias name="bob" \
     canonical_id=$(cat entity_id.txt) \
     mount_accessor=$(cat accessor_test.txt) \
     custom_metadata=account="Tester Account"

Note

To set custom metadata on the entity alias, you must run Vault 1.9 or later. If you are running Vault version earlier than v1.9, run the command without custom_metadata=account="Tester Account".

Example output:

Key             Value
---             -----
canonical_id    24204b50-22a6-61f5-bd4b-803f1a4e4726
id              ae2cdd0f-9807-7336-2265-5575c71837e7

Optional - view entity alias

To view the created entity alias, use the returned entity alias ID.

$ vault read identity/entity-alias/id/<id>

Example:

$ vault read identity/entity-alias/id/ae2cdd0f-9807-7336-2265-5575c71837e7

Key                          Value
---                          -----
canonical_id                 971b5ab3-1931-8381-8605-1b9511572278
creation_time                2021-11-15T01:41:03.093366Z
custom_metadata              map[account:Tester Account]
id                           10340e8f-64fa-fe7d-0c88-b6172293f788
last_update_time             2021-11-15T01:41:03.093366Z
local                        false
merged_from_canonical_ids    <nil>
metadata                     <nil>
mount_accessor               auth_userpass_2cbfe431
mount_path                   auth/userpass-test/
mount_type                   userpass
name                         bob
namespace_id                 root

Repeat the step to add user bsmith to the bob-smith entity. Set a custom metadata on the bob entity alias named, "account" and set its value to be "QA Eng Account".

$ vault write identity/entity-alias name="bsmith" \
     canonical_id=$(cat entity_id.txt) \
     mount_accessor=$(cat accessor_qa.txt) \
     custom_metadata=account="QA Eng Account"

Example output:

Key             Value
---             -----
canonical_id    24204b50-22a6-61f5-bd4b-803f1a4e4726
id              6066f6af-bb1c-5310-58a1-fd9c8f151573

Review the entity details.

$ vault read -format=json identity/entity/id/$(cat entity_id.txt) | jq -r ".data"

Example output: The output should include the entity aliases, metadata (organization, and team), and base policy.

{
  "aliases": [
    {
      "canonical_id": "73503625-abcd-db22-08c3-c121d682d550",
      "creation_time": "2021-11-17T05:33:48.040506Z",
      "custom_metadata": {
        "account": "Tester Account"
      },
      "id": "cf073e2e-41af-852f-848d-f67533c8a610",
      "last_update_time": "2021-11-17T05:33:48.040506Z",
      "local": false,
      "merged_from_canonical_ids": null,
      "metadata": null,
      "mount_accessor": "auth_userpass_0a6936a7",
      "mount_path": "auth/userpass-test/",
      "mount_type": "userpass",
      "name": "bob"
    },
    {
      "canonical_id": "73503625-abcd-db22-08c3-c121d682d550",
      "creation_time": "2021-11-17T05:33:48.107834Z",
      "custom_metadata": {
        "account": "QA Eng Account"
      },
      "id": "7add6763-ce53-d92a-c795-c8ae529ce6e7",
      "last_update_time": "2021-11-17T05:33:48.107834Z",
      "local": false,
      "merged_from_canonical_ids": null,
      "metadata": null,
      "mount_accessor": "auth_userpass_ef4f8068",
      "mount_path": "auth/userpass-qa/",
      "mount_type": "userpass",
      "name": "bsmith"
    }
  ],
  "creation_time": "2021-11-17T05:33:47.966585Z",
  "direct_group_ids": [
    "49e8b9e8-8933-1e14-f05c-e1c9674b142b"
  ],
  "disabled": false,
  "group_ids": [
    "49e8b9e8-8933-1e14-f05c-e1c9674b142b"
  ],
  "id": "73503625-abcd-db22-08c3-c121d682d550",
  "inherited_group_ids": [],
  "last_update_time": "2021-11-17T05:33:47.966585Z",
  "merged_entity_ids": null,
  "metadata": {
    "organization": "ACME Inc.",
    "team": "QA"
  },
  "mfa_secrets": {},
  "name": "bob-smith",
  "namespace_id": "root",
  "policies": [
    "base"
  ]
}

First, create the API request payload containing the base policy.

$ tee payload-1.json <<EOF
{
  "policy": "path \"secret/data/training_*\" {\n   capabilities = [\"create\", \"read\"]\n}"
}
EOF

Create base policy.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request PUT \
   --data @payload-1.json \
   $VAULT_ADDR/v1/sys/policies/acl/base

Create the API request payload containing the test policy.

$ tee payload-2.json <<EOF
{
  "policy": "path \"secret/data/test\" {\n capabilities = [ \"create\", \"read\", \"update\", \"delete\" ]\n }"
}
EOF

Create test policy.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request PUT \
   --data @payload-2.json \
   $VAULT_ADDR/v1/sys/policies/acl/test

Create the API request payload containing the team-qa policy.

$ tee payload-3.json <<EOF
{
  "policy": "path \"secret/data/team-qa\" {\n capabilities = [ \"create\", \"read\", \"update\", \"delete\" ]\n }"
}
EOF

Create team-qa policy.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request PUT \
   --data @payload-3.json \
   $VAULT_ADDR/v1/sys/policies/acl/team-qa

List all policies to verify that base, test and team-qa policies exist.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request LIST \
   $VAULT_ADDR/v1/sys/policies/acl | jq -r ".data"

Output:

{
  "keys": [
    "base",
    "default",
    "team-qa",
    "test",
    "root"
  ]
}

Enable the userpass auth method at userpass-test.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data '{"type": "userpass"}' \
   $VAULT_ADDR/v1/sys/auth/userpass-test

Create a new user named bob in userpass-test where the password is training and test policy is attached.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data '{"password": "training", "policies": "test"}' \
   $VAULT_ADDR/v1/auth/userpass-test/users/bob

Enable the userpass auth method at userpass-qa.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data '{"type": "userpass"}' \
   $VAULT_ADDR/v1/sys/auth/userpass-qa

Create a new user named bsmith in userpass-qa where the password is training and team-qa policy is attached.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data '{"password": "training", "policies": "team-qa"}' \
   $VAULT_ADDR/v1/auth/userpass-qa/users/bsmith

Execute the following command to discover the mount accessor for the userpass auth method enabled at userpass-test and store the value in accessor_test.txt.
```
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   $VAULT_ADDR/v1/sys/auth | jq -r '.data | .["userpass-test/"].accessor' > accessor_test.txt
```
The resulting file contains the accessor value (auth_userpass_XXXXX).
Execute the following command to discover the mount accessor for the userpass auth method enabled at userpass-qa and store the value in accessor_qa.txt.
```
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   $VAULT_ADDR/v1/sys/auth | jq -r '.data | .["userpass-qa/"].accessor' > accessor_qa.txt
```

Create an API payload to create bob-smith entity.

$ tee payload-entity.json <<EOF
{
  "name": "bob-smith",
  "metadata": {
    "organization": "ACME Inc.",
    "team": "QA"
  },
  "policies": ["base"]
}
EOF

Create bob-smith entity, and store the returned entity ID in a file named, entity_id.txt.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data @payload-entity.json \
   $VAULT_ADDR/v1/identity/entity | jq -r ".data.id" > entity_id.txt

The resulting file contains the entity ID for bob-smith (e.g. 24204b50-22a6-61f5-bd4b-803f1a4e4726).

Now, add the user bob to the bob-smith entity by creating an entity alias. In the request body, you need to pass the userpass name as name, the userpass-test accessor value as mount_accessor, and the entity id as canonical_id. Set a custom metadata on the bob entity alias named, "account" and set its value to be "Tester Account".

Create the API request payload.

$ tee payload-bob.json <<EOF
{
  "name": "bob",
  "canonical_id": "$(cat entity_id.txt)",
  "mount_accessor": "$(cat accessor_test.txt)",
  "custom_metadata": {
    "account": "Tester Account"
  }
}
EOF

Note

Use the identity/entity-alias endpoint to add an entity alias.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data @payload-bob.json \
   $VAULT_ADDR/v1/identity/entity-alias | jq -r ".data"

Example output:

{
  "canonical_id": "24204b50-22a6-61f5-bd4b-803f1a4e4726",
  "id": "ae2cdd0f-9807-7336-2265-5575c71837e7"
}

Optional - view entity alias

To view the created entity alias, use the returned entity alias ID.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   $VAULT_ADDR/v1/identity/entity-alias/id/<id> | jq -r ".data"

Example:

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   $VAULT_ADDR/v1/identity/entity-alias/id/ae2cdd0f-9807-7336-2265-5575c71837e7 \
   | jq -r ".data"

{
  "canonical_id": "65764de8-4e3a-6ea1-8c96-3df951f05b42",
  "creation_time": "2021-11-15T02:18:18.925287Z",
  "custom_metadata": {
    "account": "Tester Account"
  },
  "id": "ad67d5f0-221f-5c4a-b652-eecf107e5beb",
  "last_update_time": "2021-11-15T02:18:18.925287Z",
  "local": false,
  "merged_from_canonical_ids": null,
  "metadata": null,
  "mount_accessor": "auth_userpass_53c9a0c7",
  "mount_path": "auth/userpass-test/",
  "mount_type": "userpass",
  "name": "bob",
  "namespace_id": "root"
}

Repeat the step to add user bsmith to the bob-smith entity. Set a custom metadata on the bob entity alias named, "account" and set its value to be "QA Eng Account".

Create the API request payload.

$ tee payload-bsmith.json <<EOF
{
   "name": "bsmith",
   "canonical_id": "$(cat entity_id.txt)",
   "mount_accessor": "$(cat accessor_qa.txt)",
   "custom_metadata": {
     "account": "QA Eng Account"
   }
}
EOF

Use the identity/entity-alias endpoint to add an entity alias.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data @payload-bsmith.json \
   $VAULT_ADDR/v1/identity/entity-alias | jq -r ".data"

Example output:

{
  "canonical_id": "24204b50-22a6-61f5-bd4b-803f1a4e4726",
  "id": "6066f6af-bb1c-5310-58a1-fd9c8f151573"
}

Review the entity details.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   $VAULT_ADDR/v1/identity/entity/id/$(cat entity_id.txt) | jq -r ".data"

Example output: The bob and bsmith users should appear in the entity alias list, and the metadata associated with the entity.

{
  "aliases": [
    {
      "canonical_id": "73503625-abcd-db22-08c3-c121d682d550",
      "creation_time": "2021-11-17T05:33:48.040506Z",
      "custom_metadata": {
        "account": "Tester Account"
      },
      "id": "cf073e2e-41af-852f-848d-f67533c8a610",
      "last_update_time": "2021-11-17T05:33:48.040506Z",
      "local": false,
      "merged_from_canonical_ids": null,
      "metadata": null,
      "mount_accessor": "auth_userpass_0a6936a7",
      "mount_path": "auth/userpass-test/",
      "mount_type": "userpass",
      "name": "bob"
    },
    {
      "canonical_id": "73503625-abcd-db22-08c3-c121d682d550",
      "creation_time": "2021-11-17T05:33:48.107834Z",
      "custom_metadata": {
        "account": "QA Eng Account"
      },
      "id": "7add6763-ce53-d92a-c795-c8ae529ce6e7",
      "last_update_time": "2021-11-17T05:33:48.107834Z",
      "local": false,
      "merged_from_canonical_ids": null,
      "metadata": null,
      "mount_accessor": "auth_userpass_ef4f8068",
      "mount_path": "auth/userpass-qa/",
      "mount_type": "userpass",
      "name": "bsmith"
    }
  ],
  "creation_time": "2021-11-17T05:33:47.966585Z",
  "direct_group_ids": [
    "49e8b9e8-8933-1e14-f05c-e1c9674b142b"
  ],
  "disabled": false,
  "group_ids": [
    "49e8b9e8-8933-1e14-f05c-e1c9674b142b"
  ],
  "id": "73503625-abcd-db22-08c3-c121d682d550",
  "inherited_group_ids": [],
  "last_update_time": "2021-11-17T05:33:47.966585Z",
  "merged_entity_ids": null,
  "metadata": {
    "organization": "ACME Inc.",
    "team": "QA"
  },
  "mfa_secrets": {},
  "name": "bob-smith",
  "namespace_id": "root",
  "policies": [
    "base"
  ]
}

Open a web browser and launch the Vault UI by entering http://127.0.0.1:8200/ui in the address, and then login.
Click the Policies tab, and then select Create ACL policy.
Enter base in the Name field, and paste in the following policy in the Policy text field.
```
path "secret/data/training_*" {
   capabilities = ["create", "read"]
}
```
Click Create Policy to complete.
Select Create ACL policy again, enter test in the Name field, and paste in the following policy in the Policy text field.
```
path "secret/data/test" {
   capabilities = [ "create", "read", "update", "delete" ]
}
```
Click Create Policy to complete.
Select Create ACL policy again, enter team-qa in the Name field, and paste in the following policy in the Policy text field.
```
path "secret/data/team-qa" {
   capabilities = [ "create", "read", "update", "delete" ]
}
```
Click Create Policy to complete.
Click the Access tab, and select Enable new method.
Select the Username & Password radio button.
Click Next.
Enter userpass-test in the Path field and click Enable Method.
Click the Access tab again, and then select userpass-test/.
Select Create user. Enter bob in the Username field, and training in the Password field.
Expand the Tokens toggle.
Under Generated Token's Policies, enter test to attach test policy.
Click Save.
Select Auth Methods and select Enable new method.
Select the Username & Password radio button, and then click Next.
Enter userpass-qa in the Path field and click Enable Method.
Click the Access tab again, and then select userpass-qa/.
Select Create user. Enter bsmith in the Username field, and training in the Password field.
Expand the Tokens toggle.
Under Generated Token's Policies, enter team-qa to attach team-qa policy.
Click Save.
From the Access tab, select Entities and then Create entity to create a new entity, bob-smith. Populate the Name, Policies and Metadata fields as shown below:
Click Create.
Select Add alias. Enter bob in the Name field and select userpass-test/ (userpass) from the Auth Backend drop-down list.
Click Create.
Return to the Entities list. Select Add alias from the bob-smith entity menu, and then Add alias.
Enter bsmith in the Name field and select userpass-qa/ (userpass) from the Auth Backend drop-down list, and then click Create.
Return to the Entities > bob-smith and select Aliases. Bothbob and bsmith should be listed.

Security Consideration

Avoid storing any sensitive personally identifiable information (PII) in the entity metadata. The entity metadata is replicated to other clusters if Performance Replication is configured. This may be a great concern for the GDPR. Vault 1.9 introduced the ability to set custom metadata on each entity alias that does not overlap with the metadata set by Vault. If the auth method is local to the cluster, the metadata will not be replicated to other clusters in the same performance replication group. Therefore, it is recommended to leverage the custom metadata on the entity alias instead.

Test the entity

To better understand how a token inherits the capabilities from the entity's policy, you are going to test it by logging in as bob.

First, login as bob, and store the generated client token in bob_token.txt.

$ vault login -format=json -method=userpass -path=userpass-test \
    username=bob password=training \
    | jq -r ".auth.client_token" > bob_token.txt

The test policy grants CRUD operations on the secret/test path. Test to make sure that you can write secrets in the path.

$ VAULT_TOKEN=$(cat bob_token.txt) vault kv put secret/test owner="bob"

Key                Value
---                -----
created_time       2021-11-06T02:12:02.104146Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

Although the username bob does not have base policy attached, the token inherits the capabilities granted in the base policy because bob is a member of the bob-smith entity, and the entity has base policy attached. Check to see that the bob's token inherited the capabilities.
```
$ VAULT_TOKEN=$(cat bob_token.txt) vault token capabilities secret/data/training_test
create, read
```
The base policy grants create and read capabilities on secret/training_* path; therefore bob is permitted to run create and read operations against any path starting with secret/training_*.
What about the secret/team-qa path?
```
$ VAULT_TOKEN=$(cat bob_token.txt) vault token capabilities secret/data/team-qa
deny
```
The user bob only inherits capability from its associating entity's policy. The user can access the secret/team-qa path only if they log in with bsmith credentials.

$ curl --request POST \
   --data '{"password": "training"}' \
   $VAULT_ADDR/v1/auth/userpass-test/login/bob \
   | jq -r ".auth.client_token" > bob_token.txt

The test policy grants CRUD operations on the secret/test path. Test to make sure that you can write secrets in the path.

First, create the API request payload.

$ tee payload_test.json <<EOF
{
  "data": {
    "owner": "bob"
  }
}
EOF

Test to make sure that you can write secrets in the path.

$ curl --header "X-Vault-Token: $(cat bob_token.txt)" \
   --request POST \
   --data @payload_test.json \
   $VAULT_ADDR/v1/secret/data/test | jq -r ".data"

This should execute successfully.

Example output:

{
  "created_time": "2021-11-06T02:31:57.43338Z",
  "custom_metadata": null,
  "deletion_time": "",
  "destroyed": false,
  "version": 1
}

Although the username bob does not have base policy attached, the token inherits the capabilities granted in the base policy because bob is a member of the bob-smith entity, and the entity has base policy attached. Check to see that the bob's token inherited the capabilities.
```
$ curl --header "X-Vault-Token: $(cat bob_token.txt)" \
   --request POST \
   --data '{"paths": ["secret/data/training_test"]}' \
   $VAULT_ADDR/v1/sys/capabilities-self | jq -r ".data"
```
Output:
```
{
  "capabilities": [
    "create",
    "read"
  ],
  "secret/data/training_test": [
    "create",
    "read"
  ]
}
```
The base policy grants create and read capabilities on secret/training_* path; therefore bob is permitted to run create and read operations against any path starting with secret/training_*.

What about the secret/team-qa path?

$ curl --header "X-Vault-Token: $(cat bob_token.txt)" \
   --request POST \
   --data '{"paths": ["secret/data/team-qa"]}' \
   $VAULT_ADDR/v1/sys/capabilities-self | jq -r ".data"

Output:

{
  "capabilities": [
    "deny"
  ],
  "secret/data/team-qa": [
    "deny"
  ]
}

The user bob only inherits capability from its associating entity's policy. The user can access the secret/team-qa path only if they log in with bsmith credentials.

Create an internal group

Reset the `VAULT_TOKEN` environment variable

Log back in with the token you used to configure the entity before proceed to create an internal group. (Refer to the Lab setup section.)

Now, you are going to create an internal group named engineers. Its member is the bob-smith entity that you created in the create an entity section.

The group policy, team-eng defines the following in the team-eng.hcl file.

path "secret/data/team/eng" {
  capabilities = [ "create", "read", "update", "delete"]
}

Create a new policy named team-eng.

$ vault policy write team-eng -<<EOF
path "secret/data/team/eng" {
  capabilities = [ "create", "read", "update", "delete"]
}
EOF

Create an internal group named engineers and add bob-smith entity as a group member and attach team-eng.

$ vault write identity/group name="engineers" \
     policies="team-eng" \
     member_entity_ids=$(cat entity_id.txt) \
     metadata=team="Engineering" \
     metadata=region="North America"

Example output:

Key     Value
---     -----
id      0cd76d41-fe36-8c7b-4758-d36bbc212650
name    engineers

Now, when you login as bob or bsmith, its generated token inherits the group-level policy, team-eng. You can perform similar tests demonstrated in the test the entity section to verify that.

Create an API request payload containing the team-eng policy.

$ tee payload.json <<EOF
{
   "policy": "path \"secret/data/team/eng\" {\n  capabilities = [ \"create\", \"read\", \"update\", \"delete\"]\n}\n
}
EOF

Create a new policy named team-eng.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request PUT \
    --data @payload-1.json \
    $VAULT_ADDR/v1/sys/policies/acl/team-eng

Create an API request payload to create an internal group named engineers.

$ tee payload-group.json <<EOF
{
  "name": "engineers",
  "policies": ["team-eng"],
  "member_entity_ids": ["$(cat entity_id.txt)"],
  "metadata": {
    "team": "Engineering",
    "region": "North America"
  }
}
EOF

Use the /identity/group endpoint to create an internal group named engineers and add bob-smith entity as a group member and attach team-eng.
```
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
       --request PUT \
       --data @payload-group.json \
       $VAULT_ADDR/v1/identity/group | jq -r ".data"
```
Example output:
```
{
  "id": "3439804b-c82e-1977-a306-c9dc2e97f039",
  "name": "engineers"
}
```
Now, when you login as bob or bsmith, its generated token inherits the group-level policy, team-eng. You can perform similar tests demonstrated in the test the entity section to verify that.

Click the Policies tab, and then select Create ACL policy.

Enter team-eng in the Name field, and paste in the following policy in the Policy text field.

path "secret/data/team/eng" {
  capabilities = [ "create", "read", "update", "delete"]
}

Click Create Policy to complete.
Click the Access tab and select Groups.
Click Groups from the left navigation, and select Create group.
Enter the group information as shown below.
Click Create.

Summary

By default, Vault creates an internal group. When you create an internal group, you specify the group members rather than group alias. Group aliases are mapping between Vault and external identity providers (e.g. LDAP, GitHub, etc.). Therefore you define group aliases only when you create external groups. For internal groups, you specify member_entity_ids and/or member_group_ids.

Create an External Group

It is common for organizations to enable auth methods such as LDAP, Okta and perhaps GitHub to handle the Vault user authentication, and individual user's group memberships are defined within those identity providers.

In order to manage the group-level authorization, you can create an external group to link Vault with the external identity provider (auth provider) and attach appropriate policies to the group.

Example Scenario

Any user who belongs to training team in GitHub organization, example-inc are permitted to perform all operations against the secret/education path.

Note

This scenario assumes that the GitHub organization, example-inc exists as well as training team within the organization.

Create a new policy named education.

$ vault policy write education -<<EOF
path "secret/data/education" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}
EOF

Enable GitHub auth method.
```
$ vault auth enable github
```
Retrieve the mount accessor for the GitHub auth method and save it in a file named accessor_github.txt.
```
$ vault auth list -format=json | jq -r '.["github/"].accessor' > accessor_github.txt
```
Configure to point to your GitHub organization.
Note
Use the actual Github organization name that exists to which you want to connect. The following uses a fictional organization name, example-inc as an example. If you do not change this value, the login attempt at the end of this section will not succeed.
Example:
```
$ vault write auth/github/config organization=example-inc
```

Create an external group, and store the generated group ID in group_id.txt.

Example:

$ vault write -format=json identity/group name="education" \
     policies="education" \
     type="external" \
     metadata=organization="Product Education" | jq -r ".data.id" > group_id.txt

Create a group alias where canonical_id is the group ID and the name must be an actual GitHub team name that exists.

Note

The name must be an actual GitHub team name that exists in the Github organization that you configured earlier. Use slugified team name.

$ vault write identity/group-alias name="training" \
     mount_accessor=$(cat accessor_github.txt) \
     canonical_id="$(cat group_id.txt)"

Example output:

Key             Value
---             -----
canonical_id    66818a45-ef85-0ff3-6c1e-37faf12ea55e
id              578944f0-dcfd-29fd-a763-d3f9431512d7

Attempt to login as a GitHub user who is a member of the training group and enter the users personal access token.

$ vault login -format=json -method=github

GitHub Personal Access Token (will be hidden):

Example output:

WARNING! The VAULT_TOKEN environment variable is set! This takes precedence
over the value set by this command. To use the value set by this command,
unset the VAULT_TOKEN environment variable or set it to the token displayed
below.

{
  "request_id": "fcc10086-9033-dc21-2ddb-d420fbabc474",
  "lease_id": "",
  "lease_duration": 0,
  "renewable": false,
  "data": null,
  "warnings": null,
  "auth": {
    "client_token": "s.7QKSeasAME.mD5Ve",
    "accessor": "yUBG78LRdLiZNkBNNTe7OEp1.mD5Ve",
    "policies": [
      "default",
      "education"
    ],
    "token_policies": [
      "default"
    ],
    "identity_policies": [
      "education"
    ],
    "metadata": {
      "org": "example-inc",
      "username": "bobbysmith_developer"
    },
    "orphan": true,
    "entity_id": "58a16f6c-ee92-0cd2-f508-4b2fdb093c07",
    "lease_duration": 3600,
    "renewable": true,
    "mfa_requirement": null
  }
}

Create an API request payload containing stringified education policy.

$ tee payload-pol.json <<EOF
{
  "policy": "path \"secret/data/education\" {\n   capabilities = [\"create\", \"read\", \"update\", \"delete\", \"list\"]\n}"
}
EOF

Create education policy.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request PUT \
   --data @payload-pol.json \
   $VAULT_ADDR/v1/sys/policies/acl/education

Enable GitHub Auth Method at github.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data '{"type": "github"}' \
   $VAULT_ADDR/v1/sys/auth/github

Configure GitHub auth method by setting organization.
Note
Use the actual Github organization name that exists to which you want to connect. The following uses a fictional organization name, example-inc as an example. If you do not change this value, the login attempt at the end of this section will not succeed.
Example:
```
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data '{"organization": "example-inc"}' \
   $VAULT_ADDR/v1/auth/github/config
```

Get the github accessor value and save it in a file named accessor_github.txt.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   $VAULT_ADDR/v1/sys/auth | jq -r '.data | ."github/".accessor' > accessor_github.txt

Create an API request payload to create an external group.

Example:

$ tee payload-edu.json <<EOF
{
  "name": "education",
  "policies": ["education"],
  "type": "external",
  "metadata": {
    "organization": "Product Education"
  }
}
EOF

Create an external group, and store the generated group ID in group_id.txt.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data @payload-edu.json \
   $VAULT_ADDR/v1/identity/group | jq -r ".data.id" > group_id.txt

Create an API request payload to create a group alias for a team that exists in your Github organization.
```
$ tee payload-training.json <<EOF
{
  "canonical_id": "$(cat group_id.txt)",
  "mount_accessor": "$(cat accessor_github.txt)",
  "name": "training"
}
EOF
```
Note
The name must be an actual GitHub team name that exists in the Github organization that you configured earlier. Use slugified team name.

Create a group alias.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data @payload-training.json \
   $VAULT_ADDR/v1/identity/group-alias | jq -r ".data"

Example output:

{
   "canonical_id": "81297d74-d67e-afe6-8a57-d76769430796",
   "id": "2891f4e7-554a-2fd7-82a4-3fe890266f75"
}

Attempt to login as a GitHub user who is a member of the training group and enter the users personal access token.

$ curl --request POST \
   --data '{"token": "<actual-personal-access-token>"}'
   $VAULT_ADDR/v1/auth/github/login

Example output:

{
  "request_id": "1b3D5f-5b4b-47d5-3bb4-83b2f3cfb4a1",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": null,
  "wrap_info": null,
  "warnings": null,
  "auth": {
    "client_token": "s.IqIsLMNOPwLPjRmUVam.tj0yZ",
    "accessor": "63nn6QRSTU3ASzMoiUP4MkV2.tj0yZ",
    "policies": [
      "default",
      "education"
    ],
    "token_policies": [
      "default"
    ],
    "identity_policies": [
      "education"
    ],
    "metadata": {
      "org": "example-inc",
      "username": "bobbysmith_developer"
    },
    "lease_duration": 3600,
    "renewable": true,
    "entity_id": "f9030f4b-e310-c54b-fc0a-e7f620cd17ac",
    "token_type": "service",
    "orphan": true
  }
}

Click the Policies tab, and then select Create ACL policy.

Enter team-eng in the Name field, and paste in the following policy in the Policy text field.

path "secret/data/education" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

Click Create Policy to complete.
Click the Access tab and select Auth Methods.
Select Enable new method.
Select the GitHub radio button and click Next.
Leave the path as github and click Enable Method.
Click the Access tab and select Groups.
Select Create group. Enter the group information as shown below.
Click Create.
Select Add alias and enter training in the Name field. Select github/ (github) from the Auth Backend drop-down list.
Click Create.

Summary

At this point, any GitHub user who belongs to training team within the example-inc organization can authenticate with Vault. The generated token for the user has education policy attached.

Clean up

If you wish to clean up your environment after completing the tutorial, follow the steps in this section.

Unset the VAULT_TOKEN environment variable.
```
$ unset VAULT_TOKEN
```
Unset the VAULT_ADDR environment variable.
```
$ unset VAULT_ADDR
```
You can stop the Vault dev server by pressing Ctrl+C where the server is running. Or, execute the following command.
```
$ pgrep -f vault | xargs kill
```
For Vault Dedicated, delete the Vault instance from the HCP Portal.

Why create entities?

This tutorial used a human user, Bob Smith as an example to demonstrate the use of entities and groups. However, this concept also applies to applications.

Consider a scenario where an application uses multiple auth methods to log in with Vault. Without an entity defined to tie those multiple auth identities back to the application, the Vault server has no way of knowing the same application logged into Vault using different auth methods. As a result, each login gets counted as a new client.

This may not be an issue for Vault Community Edition users; however, this can results in unexpectedly higher client counts that affects your Vault Enterprise or Vault Dedicated billing cost.

Help and Reference

This tutorial focuses on the creation of entities and groups in Vault. Refer to Vault Limits and Maximums to understand the known upper limits when working with Vault Identities.

Retrieve secrets for Kubernetes workfloads

Build custom Vault plugins

This tutorial also appears in:

13 tutorials

Operations fundamentals
Learn basic Vault operations that are common to both Vault Community Edition and Vault Enterprise users.
- Vault