# `Once`
[🔗](https://github.com/juulSme/Once/blob/v1.3.1/lib/once.ex#L1)

Once is an Ecto type for locally unique (unique within your domain or application) 64-bit IDs generated by multiple Elixir nodes. Locally unique IDs make it easier to keep things separated, simplify caching and simplify inserting related things (because you don't have to wait for the database to return the ID).

Because a Once fits into an SQL bigint, they use little space and keep indexes small and fast. Because of their [structure](https://hexdocs.pm/no_noncense/NoNoncense.html#module-nonce-types) they have counter-like data locality, which helps your indexes perform well, [unlike UUIDv4s](https://www.cybertec-postgresql.com/en/unexpected-downsides-of-uuid-keys-in-postgresql/).

Once IDs are based on counter, time-sortable or encrypted nonces. These underlying values can then be encoded in several formats, can be prefixed and can be masked. And you can combine all of these options, providing great flexibility.

The library has only `Ecto` and its sibling `NoNoncense` as dependencies. NoNoncense generates the actual values and performs incredibly well, hitting rates of tens of millions of nonces per second, and it also helps you to safeguard the uniqueness guarantees.

> #### Read the migration guide {: .warning}
>
> If you're upgrading from v0.x.x and you use encrypted IDs, please read the [Migration Guide](MIGRATION.md) carefully - there are breaking changes that require attention to preserve uniqueness guarantees.

## Quick Start

To get going, you need to set up a `NoNoncense` instance to generate the base unique values. Follow [its documentation](https://hexdocs.pm/no_noncense) to do so. `Once` expects an instance with its own module name by default, like so:

    # application.ex (read the NoNoncense docs!)
    machine_id = NoNoncense.MachineId.id!(opts)
    NoNoncense.init(name: Once, machine_id: machine_id)

    # if you want to use encrypted/masked IDs, also pass a 256-bits key
    NoNoncense.init(name: Once, machine_id: machine_id, base_key: System.get_env("ONCE_SECRET_KEY"))

In your `Ecto` schemas, you can then use the type:

    schema "things" do
      field :id, Once, autogenerate: true
    end

And that's it!

## Core Concepts

### ID Generation Types

Once supports three types of ID generation, controlled by the `:nonce_type` option:

- **Counter** (default): Really fast to generate, predictable, works well with b-tree indexes. IDs are generated using a machine init timestamp, machine ID and counter.
- **Sortable**: Time-sortable like a [Snowflake ID](https://en.wikipedia.org/wiki/Snowflake_ID). Use this when chronological ordering is important.
- **Encrypted**: Unique and unpredictable, like a UUIDv4 but shorter. Use when you need unpredictable IDs. Note that encrypted IDs cost you data locality, decrease index performance and are slightly slower to generate. Alternatively, use [masked](#module-masked-ids) counter/sortable IDs.

### Data Formats

IDs can be represented in multiple formats, both in your Elixir application (`:ex_format`) and in your database (`:db_format`). By default, IDs are url64-encoded 11-character strings in Elixir and signed bigints in the database.

These are the formats in which an ID can be rendered.
All are equivalent and can be transformed to one another using `to_format/3`.
The examples represent the same underlying value.

- `:url64` a url64-encoded string of 11 characters, for example `"zAjhfZyAAAE"`
- `:hex` a hex-encoded string of 16 characters, for example `"cc08e17d9c800001"`
- `:raw` a binary of 8 bytes, for example `<<204, 8, 225, 125, 156, 128, 0, 1>>`
- `:signed` a signed 64-bits integer between `-(2^63)` and `2^63-1`, for example `-3744495160545771519`
- `:unsigned` an unsigned 64-bits integer between `0` and `2^64-1`, for example `14702248913163780097`
- `:hex32` an extended-hex string of 13 characters, for example `"pg4e2vcsg0002"`

With the default formats, you might see `"AAAAAACYloA"` in Elixir and `10_000_000` in your database. The reasoning behind these defaults is that the encoded format is readable, short, and JSON-safe by default, while the signed format means you can use a standard bigint column type (resulting in a small, fast index).

The negative integers will not cause problems with Postgres and MySQL - they both happily accept them. Negative integers will only start to appear after ~70 years of usage. However, be careful if you wish to sort by ID (see [Sorting Considerations](#module-sorting-considerations)).

### Prefixed IDs

You can add prefixes to IDs to make them recognizable and self-documenting. This is useful for debugging, API clarity and type safety. A prefixed ID looks like `"usr_AV7m9gAAAAU"` or `"prod_123"` - a human-readable prefix followed by the actual ID value. Use the `:prefix` option to specify the desired prefix.

Note that prefixed IDs are always binaries - using `:unsigned` or `:signed` as `:ex_format` results in numeric strings like `"prod_123"`, not raw integers. Autogenerated IDs will include the prefix.

The prefixes can optionally be persisted to the database. This is controlled by option `:persist_prefix`, the default is `false`. If the prefix is not persisted, it only exists in Elixir and is stripped before storing an ID in the database, and re-added on load. This allows us to still use 64-bit integer columns. So in Elixir you could have `"usr_AV7m9gAAAAU"` while in the database you would have `98770186085072901`.

With `persist_prefix: true`, the full prefixed string is stored. This sacrifices storage efficiency for database-level readability. Your ID would look like `"usr_AV7m9gAAAAU"` in both Elixir and your database. You can only use `db_format: :url64`, `:hex`, `:hex32` or `:raw` with `persist_prefix: true`.

Use `to_format/3` with the `:prefix` option to convert between formats while preserving the prefix.

### Encrypted IDs

If you need unpredictable IDs, you can use encrypted nonce generation. To use encrypted IDs:

- Set [option](#module-configuration) `nonce_type: :encrypted`
- Initialize `NoNoncense` with option `base_key: <some 32-byte secret binary>`
- (optional but recommended) Change the encryption algorithm using option `:cipher64` from the default `:blowfish` to `:speck` (requires optional dependency SpeckEx).

To learn more about nonce encryption and the available ciphers, see the [NoNoncense docs](https://hexdocs.pm/no_noncense/NoNoncense.html#module-encrypted-nonces).

### Masked IDs

Masked IDs provide a middle ground between plaintext and encrypted IDs. IDs are stored as plaintext in the database (preserving sequential writes and index performance) but encrypted when retrieved by the Ecto type. The application only sees encrypted values while the database maintains optimal performance.

To use masked IDs:

- Set [option](#module-configuration) `mask: true`
- Initialize `NoNoncense` with option `base_key: <some 32-byte secret binary>`
- Works with nonce types `:counter` and `:sortable` (there's no point in masking an encrypted nonce)

**Benefits**: Masked IDs have the database performance of plaintext IDs, but the application sees encrypted values. `cast/2` transparently decrypts its input, so `ORDER BY id` queries and keyset pagination work correctly — API consumers pass masked IDs and they route to the right database rows without any manual decryption. No database migration is needed for existing IDs.

**Trade-offs**: Database and application IDs look completely different (operational friction with SQL/BI tools), slight performance cost for encryption/decryption on every read/write, ordering info can leak into the application when using `ORDER BY id` queries.

## Configuration

### Options

The Ecto type takes the following optional parameters:

- `:no_noncense` name of the NoNoncense instance used to generate new IDs (default `Once`)
- `:ex_format` what an ID looks like in Elixir, one of `t:format/0` (default `:url64`). Be sure to read [the caveats](`m:Once#module-format-caveats`).
- `:db_format` what an ID looks like in your database, one of `t:format/0` (default `:signed`)
- `:nonce_type` how the nonce is generated, one of `t:nonce_type/0` (default `:counter`)
- `:mask` use encrypted IDs in Elixir; encrypt on load, decrypt on dump (default `false`)
- `:prefix` a string prefix to prepend to IDs, for example `"usr_"` or `"prod_"` (default `nil`)
- `:persist_prefix` whether to store the prefix in the database; requires a binary `:db_format` (default `false`)

### Format Caveats

Some caveats apply to the `:ex_format` option.

> #### Don't use raw integers with JS clients {: .warning}
>
> Encode `:signed` and `:unsigned` as numeric strings (e.g. `"123"`).

While JSON does not impose a precision limit on numbers, JavaScript can only represent integers up to 2^53. Once IDs encoded as integers are larger than that within 24 days of the epoch passed to NoNoncense.

> #### Using an integer format as `:ex_format` disables binary parsing, and using a binary format disables numeric string parsing {: .info}
>
> When `:ex_format` is set to an integer format (`:signed` or `:unsigned`), parsing numeric strings ("123") will be supported but casting and dumping hex-encoded, url64-encoded, hex32-encoded and raw binaries will be disabled.
>
> When `:ex_format` is set to a binary format (`:hex`, `:hex32`, `:url64` or `:raw`), those binary formats will be supported but parsing numeric strings will be disabled.
>
> Similarly, `to_format/3` only parses numeric strings as integers with option `parse_int: true`.

That's because we can't disambiguate some binaries that are valid hex, hex32, url64 and raw binaries and also valid numeric strings. An example is `"12345678901"`:

    # interpret as numeric string, format as signed int
    iex> {:ok, 12345678901}          = Once.to_format("12345678901", :signed, parse_int: true)

    # interpret as url64-encoded binary, format as signed int
    iex> {:ok, -2923406909136636083} = Once.to_format("12345678901", :signed)

The `:ex_format` setting and `:parse_int` option resolve this ambiguity, at the cost of some flexibility.

### Sorting Considerations

> #### Use format `:unsigned`, `:hex`, `:hex32` or `:raw` to sort IDs chronologically {: .info}
>
> If you want to sort IDs chronologically, avoid using `:url64`. Signed integers (`:signed`) can be used in the first ~70 years after epoch. Only `:sortable` IDs can be meaningfully sorted.

The various formats have different sorting behaviors for the same underlying value, which becomes particularly problematic with values equivalent to negative integers:

    # unsigned ints (0, max-signed, max-unsigned a.k.a. -1)
    iex> unsigned = [0, 9223372036854775807, 18446744073709551615]
    iex> ^unsigned = Enum.sort_by(unsigned, &Once.to_format!(&1, :hex))
    iex> ^unsigned = Enum.sort_by(unsigned, &Once.to_format!(&1, :hex32))
    iex> ^unsigned = Enum.sort_by(unsigned, &Once.to_format!(&1, :raw))
    iex> Enum.sort_by(unsigned, &Once.to_format!(&1, :signed))
    [18446744073709551615, 0, 9223372036854775807]                  # [max-unsigned, 0, max-signed]
    iex> Enum.sort_by(unsigned, &Once.to_format!(&1, :url64))
    [0, 18446744073709551615, 9223372036854775807]                  # [0, max-unsigned, max-signed]

IDs with `nonce_type: :encrypted` or `:counter` can't be meaningfully sorted. Encrypted IDs are effectively random, and counter IDs generated on different machines can interleave unpredictably when sorted.

Masked IDs can be meaningfully sorted in a database because they are stored in plaintext, so `ORDER BY id` works as usual. In Elixir they are encrypted, which randomizes their sort order.

#### Sorting in your database / `ORDER BY id` queries

- **PostgreSQL**: Only supports signed integers. Using `"ORDER BY id"` will work fine until you reach negative numbers (same as `:signed` in Elixir). The easiest way to deal with this is to ignore the problem and assume that a) your app will not reach that age or b) Postgres will support unsigned ints at some point in the next 70 years. Alternatively, use `db_format: :hex` or `:hex32`. These are fixed-length, human-readable formats that sort correctly throughout the range, at the cost of storage space.
- **MySQL**: Supports unsigned bigint, so `:unsigned` format works perfectly for sorting.

## Ecto Schema Integration

The recommended approach is to define a base schema module that centralises your Once configuration. This way you only need to change Once options in one place:

    defmodule MyApp.Schema do
      defmacro __using__(_) do
        quote do
          use Ecto.Schema

          @once_opts [ex_format: :hex32]
          @primary_key {:id, Once, [autogenerate: true] ++ @once_opts}
          @foreign_key_type Once
        end
      end
    end

Note that `@foreign_key_type` sets the type module but not the params (format, prefix, etc.), so `@once_opts` must also be passed to each `belongs_to`. You can merge in field-specific overrides with `++`:

    defmodule MyApp.Transaction do
      use MyApp.Schema

      schema "transactions" do
        belongs_to :user, MyApp.User, @once_opts
        belongs_to :other, MyApp.User, @once_opts ++ [on_replace: :nilify]
      end
    end

# `format`

```elixir
@type format() :: :url64 | :raw | :signed | :unsigned | :hex | :hex32
```

These are the formats in which an ID can be rendered.
All are equivalent and can be transformed to one another using `to_format/3`.
The examples represent the same underlying value.

- `:url64` a url64-encoded string of 11 characters, for example `"zAjhfZyAAAE"`
- `:hex` a hex-encoded string of 16 characters, for example `"cc08e17d9c800001"`
- `:raw` a binary of 8 bytes, for example `<<204, 8, 225, 125, 156, 128, 0, 1>>`
- `:signed` a signed 64-bits integer between `-(2^63)` and `2^63-1`, for example `-3744495160545771519`
- `:unsigned` an unsigned 64-bits integer between `0` and `2^64-1`, for example `14702248913163780097`
- `:hex32` an extended-hex string of 13 characters, for example `"pg4e2vcsg0002"`

# `init_opt`

```elixir
@type init_opt() ::
  {:no_noncense, module()}
  | {:ex_format, format()}
  | {:db_format, format()}
  | {:nonce_type, nonce_type()}
  | {:mask, boolean()}
  | {:prefix, binary() | nil}
  | {:persist_prefix, boolean()}
```

Options to initialize `Once`.

- `:no_noncense` name of the NoNoncense instance used to generate new IDs (default `Once`)
- `:ex_format` what an ID looks like in Elixir, one of `t:format/0` (default `:url64`). Be sure to read [the caveats](`m:Once#module-format-caveats`).
- `:db_format` what an ID looks like in your database, one of `t:format/0` (default `:signed`)
- `:nonce_type` how the nonce is generated, one of `t:nonce_type/0` (default `:counter`)
- `:mask` use encrypted IDs in Elixir; encrypt on load, decrypt on dump (default `false`)
- `:prefix` a string prefix to prepend to IDs, for example `"usr_"` or `"prod_"` (default `nil`)
- `:persist_prefix` whether to store the prefix in the database; requires a binary `:db_format` (default `false`)

# `nonce_type`

```elixir
@type nonce_type() :: :counter | :encrypted | :sortable
```

The way in which the underlying 64-bits nonce is generated.

See `NoNoncense` for details.

# `to_format_opt`

```elixir
@type to_format_opt() :: {:parse_int, boolean()} | {:prefix, binary()}
```

Options for `to_format/3`

- `:parse_int` parse numeric strings like `"123"`. Will give unexpected results with all-int hex/url64 inputs.
- `:prefix` format prefixed IDs (e.g. `"usr_Ad6RZCrAAAM"`) as another prefixed format (e.g. `"usr_01de91642ac00004"`)

# `to_format`

```elixir
@spec to_format(binary() | integer(), format(), [to_format_opt()]) ::
  {:ok, binary() | integer()} | :error
```

Transform the different forms that a `Once` can take to one another.
The formats can be found in `t:format/0`.

## Options

- `:parse_int` parse numeric strings like `"123"`. Will give unexpected results with all-int hex/url64 inputs.
- `:prefix` format prefixed IDs (e.g. `"usr_Ad6RZCrAAAM"`) as another prefixed format (e.g. `"usr_01de91642ac00004"`)

## Examples

    iex> id = 18446744073709551615
    iex> {:ok, "__________8" = id}                              = Once.to_format(id, :url64)
    iex> {:ok, <<255, 255, 255, 255, 255, 255, 255, 255>> = id} = Once.to_format(id, :raw)
    iex> {:ok, -1 = id}                                         = Once.to_format(id, :signed)
    iex> {:ok, "ffffffffffffffff" = id}                         = Once.to_format(id, :hex)
    iex> {:ok, "vvvvvvvvvvvvu" = id}                            = Once.to_format(id, :hex32)
    iex> {:ok, 18446744073709551615}                            = Once.to_format(id, :unsigned)

    # numeric strings are supported using :parse_int
    iex> Once.to_format("-2301195303365014983", :unsigned, parse_int: true)
    {:ok, 16145548770344536633}
    iex> Once.to_format("16145548770344536633", :hex, parse_int: true)
    {:ok, "e010831058218a39"}

    # prefixed IDs are supported using :prefix
    # note that integer formats are rendered as numeric strings so require :parse_int
    iex> id = "prfx_18446744073709551615"
    iex> {:ok, "prfx___________8" = id}                    = Once.to_format(id, :url64, prefix: "prfx_", parse_int: true)
    iex> {:ok, <<"prfx_", 18446744073709551615::64>> = id} = Once.to_format(id, :raw, prefix: "prfx_")
    iex> {:ok, "prfx_-1" = id}                             = Once.to_format(id, :signed, prefix: "prfx_")
    iex> {:ok, "prfx_ffffffffffffffff" = id}               = Once.to_format(id, :hex, prefix: "prfx_", parse_int: true)
    iex> {:ok, "prfx_vvvvvvvvvvvvu" = id}                  = Once.to_format(id, :hex32, prefix: "prfx_")
    iex> {:ok, "prfx_18446744073709551615"}                = Once.to_format(id, :unsigned, prefix: "prfx_")

# `to_format!`

```elixir
@spec to_format!(binary() | integer(), format(), [to_format_opt()]) ::
  binary() | integer()
```

Same as `to_format/3` but raises on error.

    iex> -200
    ...> |> Once.to_format!(:url64)
    ...> |> Once.to_format!(:raw)
    ...> |> Once.to_format!(:hex32)
    ...> |> Once.to_format!(:unsigned)
    ...> |> Once.to_format!(:hex)
    ...> |> Once.to_format!(:signed)
    -200

    iex> Once.to_format!(Integer.pow(2, 64), :unsigned)
    ** (ArgumentError) value could not be parsed: 18446744073709551616

    iex> "usr_AAAAAAAAAAA"
    ...> |> Once.to_format!(:unsigned, prefix: "usr_")
    ...> |> Once.to_format!(:hex, prefix: "usr_", parse_int: true)
    ...> |> Once.to_format!(:signed, prefix: "usr_")
    ...> |> Once.to_format!(:raw, prefix: "usr_", parse_int: true)
    ...> |> Once.to_format!(:hex32, prefix: "usr_")
    ...> |> Once.to_format!(:url64, prefix: "usr_")
    "usr_AAAAAAAAAAA"

---

*Consult [api-reference.md](api-reference.md) for complete listing*