Read and write most commonly used config files

Configuration files are crucial for setting up software applications, specifying settings, and managing operational parameters. In this article, we will explore the top most commonly used configuration file formats, understand their history, advantages, and disadvantages, and provide Python snippets for reading and writing these formats. These config files include JSON, conf, XML, HCL, YAML, TOML, INI, env and csv. Each section contains the history and usage, a code block for each config file, pros and cons as well as a python snippet for config file read and write.

JSON (JavaScript Object Notation)

History and Usage

JSON was introduced as a lightweight data interchange format, popularized by Douglas Crockford. It is widely used in web applications, APIs, and data interchange due to its simplicity and human-readable format.

{
  "name": "example",
  "version": "1.0",
  "settings": {
    "enabled": true,
    "timeout": 30
  }
}

• ✅Easy to read and write.
• ✅Supports hierarchical data structures.
• ✅Widely adopted across different programming languages.
• 🛑No support for comments.
• 🛑Can become verbose for complex configurations.

import json

def read_json(file_path):
    with open(file_path, 'r') as file:
        data = json.load(file)
    return data

def write_json(file_path, data):
    with open(file_path, 'w') as file:
        json.dump(data, file, indent=4)

conf file

History and Usage

Conf files are commonly used in Unix/Linux applications and web servers like Apache and Nginx. They can have varied syntax.

server {
  listen 80;
  server_name example.com;
  location / {
    proxy_pass http://localhost:3000;
  }
}

• ✅Flexible and powerful.
• ✅Can be tailored to specific needs.
• 🛑Syntax can vary widely.
• 🛑Requires specific parsing logic.

def read_conf(file_path):
    with open(file_path, 'r') as file:
        data = file.readlines()
    return data

def write_conf(file_path, data):
    with open(file_path, 'w') as file:
        file.writelines(data)

XML (eXtensible Markup Language)

History and Usage

XML, developed by the W3C in the late 1990s, is used for representing structured data. It is common in enterprise applications, web services, and configuration files for Java applications.

<config>
  <name>example</name>
  <version>1.0</version>
  <settings>
    <enabled>true</enabled>
    <timeout>30</timeout>
  </settings>
</config>

• ✅Highly structured and supports validation.
• ✅Human-readable.
• ✅Supports complex nested data structures.
• 🛑Verbose and can be difficult to read.
• 🛑Requires parsing libraries.

import xml.etree.ElementTree as ET

def read_xml(file_path):
    tree = ET.parse(file_path)
    root = tree.getroot()
    return root

def write_xml(file_path, root):
    tree = ET.ElementTree(root)
    tree.write(file_path, encoding='utf-8', xml_declaration=True)

HCL (HashiCorp Configuration Language)

History and Usage

HCL is a configuration language created by HashiCorp, used primarily in their tools like Terraform, Vault, and Consul.

resource "aws_instance" "example" {
  ami           = "ami-123456"
  instance_type = "t2.micro"
}

• ✅Designed for human readability.
• ✅Supports complex data structures.
• ✅Used in popular DevOps tools.
• 🛑Limited to HashiCorp ecosystem.
• 🛑Lacks native writing support in Python.

import hcl2

def read_hcl(file_path):
    with open(file_path, 'r') as file:
        data = hcl2.load(file)
    return data

def write_hcl(file_path, data):
    raise NotImplementedError("HCL writing is not natively supported in Python")

YAML (YAML Ain’t Markup Language)

History and Usage

YAML¹², developed in the early 2000s, is a human-readable data serialization standard. It is extensively used in configuration management tools like Ansible and orchestration tools like Kubernetes.

name: example
version: 1.0
settings:
  enabled: true
  timeout: 30

• ✅Human-readable and easy to edit.
• ✅Supports complex data structures.
• ✅Allows comments.
• 🛑Indentation-sensitive, which can lead to errors.
• 🛑Not as widely adopted as JSON.

import yaml

def read_yaml(file_path):
    with open(file_path, 'r') as file:
        data = yaml.safe_load(file)
    return data

def write_yaml(file_path, data):
    with open(file_path, 'w') as file:
        yaml.safe_dump(data, file)

TOML (Tom’s Obvious, Minimal Language)

History and Usage

TOML³, designed by Tom Preston-Werner⁴ in 2013, aims to be a minimal configuration file format that’s easy to read and write. It is used in modern programming languages like Rust.

name = "example"
version = "1.0"

[settings]
enabled = true
timeout = 30

• ✅Human-readable and writable.
• ✅Supports dates and times.
• ✅Hierarchical data support.
• 🛑Less widely adopted than JSON or YAML.

import tomllib

def read_toml(file_path):
    with open(file_path, 'r') as file:
        data = tomllib.load(file)
    return data

def write_toml(file_path, data):
    with open(file_path, 'w') as file:
        tomllib.dump(data, file)

INI (Initialization File)

History and Usage

INI files have been used since the early days of Windows for configuration purposes. They are still used in many legacy systems and simple configuration files.

[settings]
name = example
version = 1.0
enabled = true
timeout = 30

• ✅Simple key-value pairs.
• ✅Easy to read and write.
• 🛑Limited to simple configurations.
• 🛑No support for complex data structures.

import configparser

def read_ini(file_path):
    config = configparser.ConfigParser()
    config.read(file_path)
    return config

def write_ini(file_path, config):
    with open(file_path, 'w') as file:
        config.write(file)

env(Environment File)

History and Usage

Environment files are used to set environment variables and are common in Linux, Docker and CI/CD pipelines.

name=example
version=1.0
enabled=true
timeout=30

• ✅Simple key-value pairs.
• ✅Can be sourced by shell scripts.
• 🛑Limited to simple key-value pairs.
• 🛑Environment-specific.

from dotenv import load_dotenv, set_key, dotenv_values

def read_env(file_path):
    load_dotenv(file_path)
    return dotenv_values(file_path)

def write_env(file_path, data):
    with open(file_path, 'w') as file:
        for key, value in data.items():
            file.write(f"{key}={value}\n")

CSV (Comma-Separated Values)

History and Usage

CSV files are used for tabular data representation and are common in data import/export scenarios.

name,version,enabled,timeout
example,1.0,true,30

• ✅Simple and widely supported.
• ✅Easy to parse and generate.
• 🛑Limited to flat, tabular data.
• 🛑No support for nested structures.

import csv

def read_csv(file_path):
    with open(file_path, 'r') as file:
        reader = csv.reader(file)
        data = [row for row in reader]
    return data

def write_csv(file_path, data):
    with open(file_path, 'w', newline='') as file:
        writer = csv.writer(file)
        writer.writerows(data)

Conclusion

Understanding these top configuration file formats and how to work with them in Python is essential for developers and system administrators. Each format has its strengths and weaknesses, and choosing the right one depends on the specific requirements of your application or system.

With the provided Python snippets, you can easily read from and write to these configuration files, making your workflow more efficient and manageable.

1. YAML Official Website[↩]
2. The YAML Project Github[↩]
3. TOML Github[↩]
4. Tom Preston-Werner Wikipedia[↩]