Syncing Ory Kratos Identities with Other Systems

Today we are going to look at two ways of synchronising identities from Ory Kratos with another system. This can be accomplished using Kratos's webhooks functionality, but the process is not obvious. I lurk a bit in Ory's community Slack and this is one of the most common questions that comes up. So rather than repeating myself, here's the answer.

The following works for both self-hosted Kratos and Ory Network.

Use Case

Here's the scenario. You are using Ory Kratos (self hosted or through Ory Network) as your Identity Provider. In Kratos terminology, a user's account is an Identity, and Kratos uses UUIDs to identify them. You have some very basic user info in the Identity Schema, such as the user's name and email address.

Your software also needs more information about users - a Profile. This might include profile images, group memberships, whatever. This information needs to be stored as part of your app, not part of Kratos. And so you need a way of mapping Kratos identities to your users in other systems. This is especially an issue if you are migrating an existing piece of software to Ory Kratos.

The functionality needed sounds simple: when a new user creates an account through the Kratos registration flow, you need a new Profile to be created in your other system. The Profile needs to reference the Kratos Identity ID, and you would like the Profile.id to be referenced in the Kratos Identity. This gives you two way mapping.

The Easy Way

If you're building something new then you have an easy path forward. Simply set your Profile.id to a UUID, and use the value provided by Kratos. This way both Identity.id and Profile.id are the same, and the mapping can happen implicitly.

To accomplish this, you need a single Kratos webhook that calls an API endpoint with your new user details. You can use this to save a new Profile with the same ID. Job done.

A Simple Webhook Handler

We need an API POST endpoint that Kratos can call after the registration flow. A Super-simple version might look the following NestJS controller. It implements a POST /kratos-webhooks/create-profile route. This route accepts an id and name in the HTTP body, and saves a Profile to the database using TypeORM.

@Controller('kratos-webhooks')
export class KratosAfterRegistrationController {

  @Post('create-profile')
  async createProfile(@Body('id') id, @Body('name') name) {

    await this.repo.save({
      id: payload.id
      name: payload.name
      // whatever else you need to save
    });

  }
  constructor(@InjectRepository(Profile) repo: Repository<Profile>) {}
}

// for completeness, here's our Profile TypeORM entity
@Entity()
export class Profile {
  @PrimaryColumn()
  id: string;

  @Column()
  name: string;
}

Webhook Jsonnet

Kratos uses Jsonnet to allow you to shape the payload to what you need for your handler. All we need to do is extract the Identity ID, and the user's name from the Identity's traits. It might look something like this:

function(ctx) {
  id: ctx.identity.id,
  name: ctx.identity.traits.name
}

Webhook Configuration

The final step is to tell Kratos to call your webhook after registration. We need to add the following to Kratos.yaml:

registration:
  after:
    hooks:
      - hook: web_hook
        config:
          url: http://api.example.com/kratos-webhooks/create-profile
          method: POST
          body: base64://ZnVuY3Rpb24oY3R4KSB7CiAgaWQ6IGN0eC5pZGVudGl0eS5pZCwKICBuYW1lOiBjdHguaWRlbnRpdHkudHJhaXRzLm5hbWUKfQo=
          response:
            ignore: true
            parse: false 

This adds a Webhook that is triggered after registration. We provide the URL to our API endpoint. The body is a base64 encoded string of the jsonnet snippet above. I believe you can also provide a path to the jsonnet file if that works better for you.

The final response section tells Kratos what to do with the response from our endpoint. ignore: true tells Kratos that the webhook should not interrupt the registration flow. parse: false tells Kratos that it does not need to do anything with the body returned by the webhook handler.

That's It!

This webhook gets you a Profile saved to your database in your system whenever a new account is created in Kratos.

The Harder Way - Two Way Mapping

The easy way above is what I recommend for new systems. However it probably won't be possible if you are migrating to Kratos from another IDP, or any system where you already have users. In this case, all your existing users have a Profile, using some other ID. All of your existing code works with this Profile, and you can't just throw this away.

If you're game to do a migration in One Big Step, possibly with downtime, you could import all your existing users to Kratos, then update all the IDs of your Profile to match the Kratos ID. Then you could do something like the Easy Way above.

More likely is your migration to Kratos will be staged. You might be running two authentication systems in parallel for a while. You may need to continue supporting old versions of an app for quite a while until all your users have migrated. You can't just switch from one to another.

Don't dispair though, you can still accomplish two way mapping, just with two webhooks instead of one.

1. The first webhook happens before Kratos saves the identity to it's database. We use this webhook to save a Profile, using whatever ID scheme we already have. We pass this back to Kratos through either the metadata_admin or metadata_public object.
2. The second webhook happens after save, where we will have the Kratos Identity ID. The second webhook updates our Profile with the Kratos ID

Using this approach we can go from Profile -> Identity by using profile.kratosId and from Identity -> Profile using identity.metadata_public.profileId.

Webhook 1: Create Our Profile, Send Our ID to Kratos

The first webhook is a before-save, flow interrupting webhook. Our webhook gets called after the user has submitted their registration, but before Kratos writes the identity to the database. Kratos allows us to modify the Identity before it saves it. As the identity hasn't been saved yet, the Kratos ID won't be valid (it will be all zeros).

This looks similar to before, except we are using our own ID scheme, and returning a body from our webhook:

@Controller('kratos-webhooks')
export class CreateProfileController {

  @Post('create-profile')
  async createProfile(@Body('id') id, @Body('name') name) {

    const profile = await this.repo.save({
      name: payload.name
    });
    return {
      identity: {
        metadata_public: {
          profile_id: member.id
        }
      }
    };
  }
  
  constructor(@InjectRepository(Profile) repo: Repository<Profile>) {}
}

// for completeness, here's our Profile TypeORM entity
@Entity()
export class Profile {
  // in this example we generate our own ID
  @PrimaryGeneratedColumn('uuid')
  id: string;

  // This column will store the Kratos ID
  @Column({type: 'uuid', unique: true, nullable: true})
  kratosId: string;

  @Column()
  name: string;
}

In the response we are returning metadata_public. Kratos then saves this along with the rest of the identity to its database.

The kratos.yaml looks similar to before, except we set ignore and parse appropriately:

registration:
  after:
    hooks:
      - hook: web_hook
        config:
          url: http://api.example.com/kratos-webhooks/create-profile
          method: POST
          body: base64://ZnVuY3Rpb24oY3R4KSB7CiAgaWQ6IGN0eC5pZGVudGl0eS5pZCwKICBuYW1lOiBjdHguaWRlbnRpdHkudHJhaXRzLm5hbWUKfQo=
          response:
            ignore: false
            parse: true 

Webhook 2: Update our Profile with the Kratos ID

Kratos calls Webhook 1, then saves the identity, and then calls our second webhook. This webhook can then update our previously saved profile with the Kratos ID:

@Controller('kratos-webhooks')
export class UpdateKratosIdController {

  @Post('update-profile')
  async updateProfile(@Body('id') id, @Body('profileId') profileId) {

    const profile = await this.repo.update(
    {
      id: profileId
    },
    {
      kratosId: payload.id
    });
  }
  
  constructor(@InjectRepository(Profile) repo: Repository<Profile>) {}
}

We need a slightly different jsonnet here, because we need to get the profile ID out of metadata_public:

function(ctx) {
  id: ctx.identity.id,
  profileId: ctx.identity.metadata_public.profile_id
}

This webhook can be an async/non-interrupting one like in The Easy Way:

registration:
  after:
    hooks:
      - hook: web_hook
        config:
          url: http://api.example.com/kratos-webhooks/update-profile
          method: POST
          body: base64://ZnVuY3Rpb24oY3R4KSB7CiAgaWQ6IGN0eC5pZGVudGl0eS5pZCwKICBwcm9maWxlSWQ6IGN0eC5pZGVudGl0eS5tZXRhZGF0YV9wdWJsaWMucHJvZmlsZV9pZAp9Cg==
          response:
            ignore: true
            parse: false

Things You Should Do

Webhook Config for each registration type:

It's not well documented, but you may need to copy your webhook config into each registration type (password, code, etc) subbranch of your kratos.yaml. If you find your webhooks not being called, that's probably why, and you'll need something like this:

registration:
  after:
    hooks:
      - hook: web_hook
        config:
          url: http://api.example.com/kratos-webhooks/create-profile
          method: POST
          body: base64://YOUR_PAYLOAD
          response:
            ignore: true
            parse: false 
    password:
      hooks:
        - hook: web_hook
          config:
            url: http://api.example.com/kratos-webhooks/create-profile
            method: POST
            body: base64://YOUR_PAYLOAD
            response:
              ignore: true
              parse: false 

Webhook Security

Kratos provides a couple of different ways to secure webhooks. You don't want the API endpoint to be called by just anyone. Kratos allows you to configure API Key (shared secret) or HTTP Basic Auth, so you can lock down your endpoint.

You 100% must do this if your API endpoint is available over the public internet. If you have Kratos and your API running in a private network, and your endpoint is only available privately, you may not need it. I am not in charge of your security!

Public or Admin Metadata

Kratos provides two locations for you to add arbitrary data to your identity: metadata_public and metadata_admin. You'll need to decide which is best suited to your needs.

Basically it comes down to whether you want your client to be able to read it. If yes, then it should go in metadata_public, which is returned with toSession. metadata_admin can only be read from the Admin API. You cannot use traits because the user can edit these properties through the settings flows!

Conclusion

You absolutely can use Kratos webhooks to synchronise and map Kratos identities to other systems. You can do this to make migrating existing production systems to Kratos much easier. And now you know how to do it!

(corrections, comments, etc. welcome)

Appendix

sequenceDiagram
        actor User
        participant Kratos
        participant Webhook1
        participant Webhook2
        User->>+Kratos: Submit Registration Flow
        Kratos->>+Webhook1: Send name, etc
        Webhook1->>-Kratos: Profile ID
        Kratos->>+Webhook2: Identity ID
        deactivate Webhook2
        Kratos->>-User: Registration Success