Quick Start

Create an option type

Create an option type with macro @option as following

@option struct YouOptionType <: YourAbstractType
   a::Int = 1
   b::Float64 # required field
end

then you can use this as an option type, it can let you:

  1. convert an option type defined in Julia to a markup language, such as TOML, JSON
  2. read from plain AbstractDict{String}, TOML, JSON etc. and convert the data to the option type
  3. compose several option types together

You can easily create hierarchical struct types as following

julia> "Option A"
       @option "option_a" struct OptionA
           name::String
           int::Int = 1
       end

julia> "Option B"
       @option "option_b" struct OptionB
           opt::OptionA = OptionA(;name = "Sam")
           float::Float64 = 0.3
       end

and convert a dict to an option type via from_dict.

julia> d = Dict{String, Any}(
           "opt" => Dict{String, Any}(
               "name" => "Roger",
               "int" => 2,
           ),
           "float" => 0.33
       );

julia> option = from_dict(OptionB, d)
OptionB(;
    opt = OptionA(;
        name = "Roger",
        int = 2,
    ),
    float = 0.33,
)

when there are multiple possible option type for one field, one can use the alias to distinguish them

julia> @option struct OptionD
           opt::Union{OptionA, OptionB}
       end

julia> d1 = Dict{String, Any}(
               "opt" => Dict{String, Any}(
                   "option_b" => d
               )
           );

julia> from_dict(OptionD, d1)
OptionD(;
    opt = OptionB(;
        opt = OptionA(;
            name = "Roger",
            int = 2,
        ),
        float = 0.33,
    ),
)

julia> using Configurations

julia> @option struct OptionA
           name::String
           int::Int = 1
       end

julia> @option struct OptionB
           opt::OptionA = OptionA(;name = "Sam")
           float::Float64 = 0.3
       end

julia> d = Dict(
           "opt" => Dict(
               "name" => "Roger",
               "int" => 2,
           ),
           "float" => 0.33
       )
Dict{String, Any} with 2 entries:
  "opt"   => Dict{String, Any}("int"=>2, "name"=>"Roger")
  "float" => 0.33

julia> option = from_dict(OptionB, d)
OptionB(;
  opt = OptionA(;
    name = "Roger",
    int = 2,
  ),
  float = 0.33,
)

Or you can also create it from keyword arguments, e.g

julia> from_kwargs(OptionB; opt_name="Roger", opt_int=2, float=0.33)
OptionB(;
    opt = OptionA(;
        name = "Roger",
        int = 2,
    ),
    float = 0.33,
)

for option types you can always convert AbstractDict to a given option type, or convert them back to dictionary via to_dict, e.g

julia> Configurations.to_dict(option)
OrderedDict{String, Any} with 2 entries:
  "opt"   => OrderedDict{String, Any}("name"=>"Roger", "int"=>2)
  "float" => 0.33

for serialization, you can use the builtin TOML support

julia> to_toml(option)
"float = 0.33\n\n[opt]\nname = \"Roger\"\nint = 2\n"

Or serialize it to other format from OrderedDict.

The Reflect Type

One can use Reflect type to denote a field contains the type information of the struct.

Configurations.ReflectType
Reflect

Placeholder type for reflected type string.

Type Alias

if the corresponding type has a type_alias defined, serialization and parsing will use the type_alias instead of the type name, this only works on concrete types since the alias cannot contain any type var information.

Example

the following option struct

@option struct MyOption
    type::Reflect
    name::String = "Sam"
end

would be equivalent to

type = "MyOption"
name = "Sam"

this is useful for defining list of different types etc.

source

This is useful when you have a few different option type for one field, e.g

@option struct Option
   field::Union{OptionA, OptionB, OptionC}
end

the type information of different type will be embeded in the corresponding string of the Reflect field.

Create pretty printing for your option type

One can overload the Base.show method to create your own pretty printing. However, if you are fine with the printing style provided by GarishPrint, you can simply define the following

# using GarishPrint
Base.show(io::IO, ::MIME"text/plain", x::MyOption) = GarishPrint.pprint_struct(io, x)

This will enable pretty printing provided by GarishPrint when a rich text environment is available, and it will fallback to the default julia printing if "text/plain" MIME type is not available.

Read from Keyword Arguments

Option types can be used to organize large number of keyword arguments, and one can also construct an option type from keyword arguments via from_kwargs

Configurations.from_kwargsFunction
from_kwargs(convention!, ::Type{T}; kw...) where T

Convert keyword arguments to given option type T using convention!. See also from_dict.

Convention

  • from_underscore_kwargs!: use _ to disambiguate subfields of the same name, this is the default behaviour.
  • from_field_kwargs!: do not disambiguate subfields, errors if there are disambiguity
source
from_kwargs(::Type{T}; kw...) where T

Convert keyword arguments to given option type T using the underscore convention.

source

A real world example is the Pluto configuration.

Modify Parsed Configurations

In some cases, user may want to change a few entries after the configuration get parsed. This can be done via the keyword arguments of from_dict or from_toml.

@option struct SubOption
   field::Int
end

@option struct MyOption
   option::SubOption
end

by default from_dict or from_toml uses the from_underscore_kwargs! convention, so one can change the field entry in SubOption above via

from_dict(d; option_field=2)

Read from TOML files

Configurations supports TOML file by default via the TOML standard library, you can directly read a TOML file to your option types via from_toml.

Configurations.from_tomlFunction
from_toml(::Type{T}, filename::String; kw...) where T

Convert a given TOML file filename to an option type T. Valid fields can be override by keyword arguments. See also from_dict.

source

Read from YAML files

You can use the JuliaData/YAML package to parse a YAML file to a Dict{String, Any},

julia> using YAML, Configurations

julia> @option struct MyOption
           a::Int
           b::Float64
       end

julia> data = YAML.load_file("test.yml"; dicttype=Dict{String, Any})
Dict{String, Any} with 2 entries:
  "b" => 2
  "a" => 1

julia> from_dict(MyOption, data)
MyOption(1, 2.0)

but remember to tell the YAML parser that you would like the keys to be String since from_dict expects the dictionary to be AbstractDict{String}, this can be done via dicttype keyword as above example.

Read JSON files

One can read JSON files as a dictionary via JuliaIO/JSON.

julia> using JSON

julia> d = JSON.parse("{\"a\":1,\"b\":2.1}")
Dict{String, Any} with 2 entries:
  "b" => 2.1
  "a" => 1

julia> from_dict(MyOption, d)
MyOption(1, 2.1)

or for JSON3 you can use the following

julia> using JSON3, Configurations

julia> @option struct OptionA
        x::String = "hi"
        y::Vector{String} = String[]
           end

julia> d = JSON3.read("""
           {"y": ["a"]}
           """, Dict{String, Any})
Dict{String, Any} with 1 entry:
  "y" => Any["a"]

julia> from_dict(OptionA, d)
OptionA("hi", ["a"])

Read other formats

For other formats, as long as you can convert them to a subtype of AbstractDict{String}, you can always convert it to the option type you just defined via from_dict, however for the sake of simplicity Configurations will not ship such functionality with it.

Write to TOML

To write the option struct to a TOML file, simply use the to_toml function

julia> to_toml(option; include_defaults=false) # write to a String

julia> to_toml("test.toml", option; include_defaults=false) # write to a file

You may also be interested in the docstring of to_toml

Configurations.to_tomlFunction
to_toml([f::Function], io::IO, option; sorted=false, by=identity, kw...)

Convert an instance option of option type to TOML and write it to IO. See to_dict for other valid keyword options. See also TOML.print in the stdlib for the explaination of sorted, by and f.

Exclude nothing

In TOML specification, there is no null type. One should exclude the field if it is not specified (of value nothing in Julia). In to_toml the option exclude_nothing is always true.

In most cases, nothing is used with another type to denote optional or not specified field, thus one should always put a default value nothing to the option struct, e.g

One should define

@option struct OptionX
    a::Union{Nothing, Int} = nothing
    b::Maybe{Int} = nothing
end

Here Maybe{T} is a convenient alias of Union{Nothing, T}.

source
to_toml([f::Function], filename::String, option; sorted=false, by=identity, kw...)

Convert an instance option of option type to TOML and write it to filename. See also TOML.print.

source
to_toml(x; sorted=false, by=identity, kw...)

Convert an instance x of option type to TOML and write it to String. See also TOML.print.

to_toml does not export fields that are of the same values as the defaults. This can be overridden by changing include_defaults to true.

source

Write to YAML

To write the option struct to other formats, you need to convert it to a dictionary type first via to_dict

Configurations.to_dictFunction
to_dict(x; include_defaults=true, exclude_nothing=false) -> OrderedDict

Convert an object x to an OrderedDict.

Kwargs

  • include_defaults: include the default value, default is true.
  • exclude_nothing: exclude fields that have value nothing, this supersedes include_defaults when they are both true.

Format Compatibilty

When mapping an option struct from Julia to TOML/YAML/JSON/etc. format, there are some subtle semantic compatibilty one need to deal with, we provide some convenient predefined conversion option constants as TOMLStyle, YAMLStyle, JSONStyle.

Tips

to_dict does not export fields that are of the same values as the defaults. In most cases, this should be the default behaviour, and users should not use include_defaults, however, this can be overridden by changing include_defaults to true.

source
to_dict(x, option::ToDictOption) -> OrderedDict

Convert an object x to an OrderedDict with ToDictOption specified.

Example

to_dict(x, TOMLStyle) # TOML compatible
to_dict(x, YAMLStyle) # YAML compatible
to_dict(x, JSONStyle) # JSON compatible
source
to_dict(::Type{T}, x, option::ToDictOption) where T

Convert x when x is inside an option type T. option is a set of options to determine the conversion behaviour. this can be overloaded to change the behaviour of to_dict(x; kw...).

to_dict(::Type{T}, x) where T

One can also use the 2-arg version when x is not or does not contain an option type for convenience.

Example

The following is a builtin overload to handle list of options.

function Configurations.to_dict(::Type{T}, x::Vector, option::ToDictOption) where T
    if is_option(eltype(x))
        return map(p->to_dict(T, p, include_defaults), x)
    else
        return x
    end
end

The following overloads the 2-arg to_dict to convert all VersionNumber to a String for all kinds of option types.

Configurations.to_dict(::Type, x::VersionNumber) = string(x)
source

Then you can use YAML package to write the dict to a YAML file

julia> using YAML, Configurations

julia> d = to_dict(your_option, YAMLStyle)

julia> YAML.write_file("myfile.yaml", d)

Write to JSON

or for JSON, we recommend using JSON or JSON3 to write the file as following

for JSON

julia> using JSON, Configurations

julia> d = to_dict(your_option, JSONStyle)

julia> open("file.json", "w") do f
           JSON.print(f, d)
       end

for JSON3 you can use the following code snippet

julia> using JSON3, Configurations

julia> @option struct OptionA
        x::String = "hi"
        y::Vector{String} = String[]
    end

julia> d = to_dict(OptionA(y=["a"]))

julia> JSON3.write(d)
"{\"x\":\"hi\",\"y\":[\"a\"]}"

julia> JSON3.write("file.json", d) # write to a file
"file.json"

Write to other formats

For other formats, you can convert your option struct to an OrderedDict{String, Any} via to_dict then serialize the dictionary to your desired format.

Work with StructTypes and JSON3

One can work with StructType with Configurations to make JSON.read(json_string, MyOptionType) work automatically by copying the following code and replace MyOptionType with your own option struct types.

using StructTypes
using Configurations
StructTypes.StructType(::Type{<:MyOptionType}) = StructTypes.CustomStruct()
StructTypes.lower(x::MyOptionType) = to_dict(x, JSONStyle)
StructTypes.lowertype(::Type{<:MyOptionType}) = OrderedDict{String, Any}
StructTypes.construct(::Type{T}, x) where {T <: MyOptionType} = from_dict(T, x)

then JSON.read("// a json string or IO", MyOptionType) will just work.

Type Conversion

Since markup languages usually do not support arbitrary Julia types, thus, one may find the from_dict complain that cannot convert an object of type XXX to an object of type YYY. Usually this is because you haven't overload Base.convert from XXX to YYY for the custom struct type, usually this can be resolved via the following overload

Base.convert(::Type{MyType}, x::String) = MyType(x)

where we assume you have written a constructor from String here.

However, in some cases, you may want to do the conversion only for one OptionType without causing type piracy, for example, one may want to convert all the String to Symbol for MyOption, this can be done by overloading Configurations.from_dict

Configurations.from_dict(::Type{MyOption}, ::Type{Symbol}, s) = Symbol(s)

For more detailed type conversion mechanism, please read the Type Conversion section.