StackExchange/dnscontrol
1
1
Fork 0
mirror of https://github.com/StackExchange/dnscontrol.git synced 2025-09-04 20:24:23 +08:00
Code Issues Projects Releases Packages Wiki Activity

promote home page

Browse source
This commit is contained in:
Craig Peterson 2017-10-02 13:35:13 -04:00
parent 373b4e4b83
commit 6df4396d93
4 changed files with 147 additions and 483 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

172
docs/index.md
View file

@ -1,41 +1,163 @@
---
layout: default
---
# DNSControl
Dnscontrol is a platform for seamlessly managing your dns configuration across any number of DNS hosts, both in the cloud or in your own infrastructure. It manages all of the domains for the Stack Exchange network.
<div class="row jumbotron">
<div class="col-md-12">
<div>
<h1 class="hometitle">DnsControl</h1>
<p class="lead">DnsControl is a platform for seamlessly managing your DNS configuration across any number of DNS hosts, both in the cloud or in your own infrastructure. It manages all of the domains for the Stack Overflow network, and can do the same for you!</p>
</div>
</div>
</div>
## Getting Started
<div class="row text-center" style="padding-top: 75px;">
<div class="col-md-4">
<h3>Try It</h3>
<p>Want to jump right in? Follow our
<strong><a href="getting-started">quick start tutorial</a></strong>
on a new domain or
<strong><a href="migrating">migrate</a></strong>
an existing one. Read the
<strong><a href="js">language spec</a></strong>
for more info.
</p>
</div>
### [Getting Started]({{site.github.url}}/getting-started): A walk-through of the basics.
<div class="col-md-4">
<h3>Use It</h3>
<p>Take advantage of the
<strong><a href="">advanced features</a></strong>.
Use macros and variables for easier updates.
<!-- Optimize your SPF records. -->
Upload your zones to
<strong><a href="provider-list">multiple DNS providers</a></strong>.
</p>
</div>
### [Providers]({{site.github.url}}/provider-list): Which DNS providers are supported.
<div class="col-md-4">
<h3>Get Involved</h3>
<p>Join our
<strong><a href="https://groups.google.com/forum/#!forum/dnscontrol-discuss">mailing list</a></strong>.
We make it easy to contribute by using
<strong><a href="https://github.com/StackExchange/dnscontrol">GitHub</a></strong>,
you can make code changes with confidence thanks to extensive integration tests.
The project is
<strong><a href="https://everythingsysadmin.com/2017/08/go-get-up-to-speed.html">newbie-friendly</a></strong>
so jump right in!
</p>
</div>
</div>
### [Examples]({{site.github.url}}/examples): The DNSControl language by example.
<div class="row" style="padding-top: 75px"><div class='col-md-4 col-md-offset-4'><h2 class="text-center feature-header">Features</h2></div></div>
<hr class="feature">
### [Migrating]({{site.github.url}}/migrating): Migrating zones to DNSControl.
<div class="row">
{% include feature.html text="Maintain your DNS data as a high-level DS, with macros, and variables for easier updates." img="biology.svg" %}
{% include feature.html text="Super extensible! Plug-in architecture makes adding new DNS providers and Registrars easy!" img="light-bulb.svg" %}
{% include feature.html text="Eliminate vendor lock-in. Switch DNS providers easily, any time, with full fidelity." img="group.svg" %}
{% include feature.html text="Reduce points of failure: Easily maintain dual DNS providers and easily drop one that is down." img="layers.svg" %}
{% include feature.html text="Supports 10+ DNS Providers including BIND, AWS Route 53, Google DNS, and name.com" img="cancel.svg" %}
{% include feature.html text="Apply CI/CD principles to DNS: Unit-tests, system-tests, automated deployment." img="share.svg" %}
{% include feature.html text="All the benefits of Git (or any VCS) for your DNS zone data. View history. Accept PRs." img="document.svg" %}
{% include feature.html text="Optimize DNS with SPF optimizer. Detect too many lookups. Flatten includes." img="mail.svg" %}
{% include feature.html text="Runs on Linux, Windows, Mac, or any operating system supported by Go." img="speech-bubble.svg" %}
{% include feature.html text="Enable/disable Cloudflare proxying (the \"orange cloud\" button) directly from your DNSControl files." img="cloud-computing.svg" %}
{% include feature.html text="Assign an IP address to a constant and use the variable name throughout the configuration. Need to change the IP address globally? Just change the variable and \"recompile.\"" img="compass.svg" %}
{% include feature.html text="Keep similar domains in sync with transforms, macros, and variables." img="attachment.svg" %}
</div>
<hr class="feature">
## Reference
<div class="container-fluid">
<div class="row">
<div class="col-md-4">
<h2>
Getting Started
</h2>
<p>
Information for new users and the curious.
</p>
### [Language Reference]({{site.github.url}}/js): Description of the DNSControl language (DSL).
<ul>
<li>
<a href="{{site.github.url}}/getting-started">Getting Started</a>: A walk-through of the basics
</li>
<li>
<a href="{{site.github.url}}/provider-list">Providers</a>: Which DNS providers are supported
</li>
<li>
<a href="{{site.github.url}}/examples">Examples</a>: The DNSControl language by example
</li>
<li>
<a href="{{site.github.url}}/migrating">Migrating</a>: Migrating zones to DNSControl
</li>
</ul>
</div>
<div class="col-md-4">
<h2>
Reference
</h2>
<p>
Language resources and procedures.
</p>
### [ALIAS / ANAME records in dnscontrol]({{site.github.url}}/alias)
<ul>
<li>
<a href="{{site.github.url}}/js">Language Reference</a>: Description of the entire language
</li>
<li>
<a href="{{site.github.url}}/alias">ALIAS / ANAME records in dnscontrol</a>
</li>
<li>
<a href="{{site.github.url}}/spf">SPF Optimizer</a>: Optimize your SPF records
</li>
</ul>
</div>
<div class="col-md-4">
<h2>
Advanced Topics
</h2>
<p>
Take advantage of DNSControl's unique features.
</p>
<ul>
<li>
<a href="">Why CNAME/MX/NS targets require a trailing "dot{{site.github.url}}/why-the-dot"</a>
</li>
<li>
<a href="{{site.github.url}}/unittests">Testing</a>: Unit Testing for you DNS Data
</li>
### [Why CNAME/MX/NS targets require a trailing "dot"]({{site.github.url}}/why-the-dot)
</ul>
</div>
</div>
<div class="row">
<div class="col-md-12">
<h2>
Developer Info
</h2>
<p>
It is easy to add features and new providers to DNSControl. The code is very modular and easy to modify. There are extensive integration tests that make it easy to boldly make changes with confidence that you'll know if anything is broken. Our mailing list is friendly. Afraid to make your first PR? We'll gladly mentor you through the process. Many major code contributions have come from <a href="https://everythingsysadmin.com/2017/08/go-get-up-to-speed.html">first-time Go users</a>!
</p>
<ul>
<li>
Github: <a href="https://github.com/StackExchange/dnscontrol">https://github.com/StackExchange/dnscontrol</a>
</li>
<li>
Mailing list: <a href="https://groups.google.com/forum/#!forum/dnscontrol-discuss">dnscontrol-discuss</a>: The friendly best place to ask questions and propose new features
</li>
<li>
<a href="{{site.github.url}}/writing-providers">Step-by-Step Guide: Writing Providers</a>: How to write a DNS or Registrar Provider
</li>
<li>
<a href="{{site.github.url}}/adding-new-rtypes">Step-by-Step Guide: Adding new DNS rtypes</a>: How to add a new DNS record type
</li>
</ul>
</div>
</div>
</div>
<hr class="feature">
## Advanced Usage
### [Testing]({{site.github.url}}/unittests): Unit Testing for you DNS Data.
## Developer info
### [github](https://github.com/StackExchange/dnscontrol): Get the source!
### [Writing Providers]({{site.github.url}}/writing-providers)
### [Adding new DNS record types]({{site.github.url}}/adding-new-rtypes)
(Preview the new [frontpage]({{site.github.url}}/new-index))
<p><small>Icons made by Freepik from <a href="http://www.flaticon.com">www.flaticon.com</a></small></p>

163
docs/new-index.md
View file

@ -1,163 +0,0 @@
---
layout: default
---
<div class="row jumbotron">
<div class="col-md-12">
<div>
<h1 class="hometitle">DnsControl</h1>
<p class="lead">DnsControl is a platform for seamlessly managing your DNS configuration across any number of DNS hosts, both in the cloud or in your own infrastructure. It manages all of the domains for the Stack Overflow network, and can do the same for you!</p>
</div>
</div>
</div>
<div class="row text-center" style="padding-top: 75px;">
<div class="col-md-4">
<h3>Try It</h3>
<p>Want to jump right in? Follow our
<strong><a href="getting-started">quick start tutorial</a></strong>
on a new domain or
<strong><a href="migrating">migrate</a></strong>
an existing one. Read the
<strong><a href="js">language spec</a></strong>
for more info.
</p>
</div>
<div class="col-md-4">
<h3>Use It</h3>
<p>Take advantage of the
<strong><a href="">advanced features</a></strong>.
Use macros and variables for easier updates.
<!-- Optimize your SPF records. -->
Upload your zones to
<strong><a href="provider-list">multiple DNS providers</a></strong>.
</p>
</div>
<div class="col-md-4">
<h3>Get Involved</h3>
<p>Join our
<strong><a href="https://groups.google.com/forum/#!forum/dnscontrol-discuss">mailing list</a></strong>.
We make it easy to contribute by using
<strong><a href="https://github.com/StackExchange/dnscontrol">GitHub</a></strong>,
you can make code changes with confidence thanks to extensive integration tests.
The project is
<strong><a href="https://everythingsysadmin.com/2017/08/go-get-up-to-speed.html">newbie-friendly</a></strong>
so jump right in!
</p>
</div>
</div>
<div class="row" style="padding-top: 75px"><div class='col-md-4 col-md-offset-4'><h2 class="text-center feature-header">Features</h2></div></div>
<hr class="feature">
<div class="row">
{% include feature.html text="Maintain your DNS data as a high-level DS, with macros, and variables for easier updates." img="biology.svg" %}
{% include feature.html text="Super extensible! Plug-in architecture makes adding new DNS providers and Registrars easy!" img="light-bulb.svg" %}
{% include feature.html text="Eliminate vendor lock-in. Switch DNS providers easily, any time, with full fidelity." img="group.svg" %}
{% include feature.html text="Reduce points of failure: Easily maintain dual DNS providers and easily drop one that is down." img="layers.svg" %}
{% include feature.html text="Supports 10+ DNS Providers including BIND, AWS Route 53, Google DNS, and name.com" img="cancel.svg" %}
{% include feature.html text="Apply CI/CD principles to DNS: Unit-tests, system-tests, automated deployment." img="share.svg" %}
{% include feature.html text="All the benefits of Git (or any VCS) for your DNS zone data. View history. Accept PRs." img="document.svg" %}
{% include feature.html text="Optimize DNS with SPF optimizer. Detect too many lookups. Flatten includes." img="mail.svg" %}
{% include feature.html text="Runs on Linux, Windows, Mac, or any operating system supported by Go." img="speech-bubble.svg" %}
{% include feature.html text="Enable/disable Cloudflare proxying (the \"orange cloud\" button) directly from your DNSControl files." img="cloud-computing.svg" %}
{% include feature.html text="Assign an IP address to a constant and use the variable name throughout the configuration. Need to change the IP address globally? Just change the variable and \"recompile.\"" img="compass.svg" %}
{% include feature.html text="Keep similar domains in sync with transforms, macros, and variables." img="attachment.svg" %}
</div>
<hr class="feature">
<div class="container-fluid">
<div class="row">
<div class="col-md-4">
<h2>
Getting Started
</h2>
<p>
Information for new users and the curious.
</p>
<ul>
<li>
<a href="{{site.github.url}}/getting-started">Getting Started</a>: A walk-through of the basics
</li>
<li>
<a href="{{site.github.url}}/provider-list">Providers</a>: Which DNS providers are supported
</li>
<li>
<a href="{{site.github.url}}/examples">Examples</a>: The DNSControl language by example
</li>
<li>
<a href="{{site.github.url}}/migrating">Migrating</a>: Migrating zones to DNSControl
</li>
</ul>
</div>
<div class="col-md-4">
<h2>
Reference
</h2>
<p>
Language resources and procedures.
</p>
<ul>
<li>
<a href="{{site.github.url}}/js">Language Reference</a>: Description of the entire language
</li>
<li>
<a href="{{site.github.url}}/alias">ALIAS / ANAME records in dnscontrol</a>
</li>
<li>
<a href="{{site.github.url}}/spf">SPF Optimizer</a>: Optimize your SPF records
</li>
</ul>
</div>
<div class="col-md-4">
<h2>
Advanced Topics
</h2>
<p>
Take advantage of DNSControl's unique features.
</p>
<ul>
<li>
<a href="">Why CNAME/MX/NS targets require a trailing "dot{{site.github.url}}/why-the-dot"</a>
</li>
<li>
<a href="{{site.github.url}}/unittests">Testing</a>: Unit Testing for you DNS Data
</li>
</ul>
</div>
</div>
<div class="row">
<div class="col-md-12">
<h2>
Developer Info
</h2>
<p>
It is easy to add features and new providers to DNSControl. The code is very modular and easy to modify. There are extensive integration tests that make it easy to boldly make changes with confidence that you'll know if anything is broken. Our mailing list is friendly. Afraid to make your first PR? We'll gladly mentor you through the process. Many major code contributions have come from <a href="https://everythingsysadmin.com/2017/08/go-get-up-to-speed.html">first-time Go users</a>!
</p>
<ul>
<li>
Github: <a href="https://github.com/StackExchange/dnscontrol">https://github.com/StackExchange/dnscontrol</a>
</li>
<li>
Mailing list: <a href="https://groups.google.com/forum/#!forum/dnscontrol-discuss">dnscontrol-discuss</a>: The friendly best place to ask questions and propose new features
</li>
<li>
<a href="{{site.github.url}}/writing-providers">Step-by-Step Guide: Writing Providers</a>: How to write a DNS or Registrar Provider
</li>
<li>
<a href="{{site.github.url}}/adding-new-rtypes">Step-by-Step Guide: Adding new DNS rtypes</a>: How to add a new DNS record type
</li>
</ul>
</div>
</div>
</div>
<hr class="feature">
<p><small>Icons made by Freepik from <a href="http://www.flaticon.com">www.flaticon.com</a></small></p>

101
docs/new-toc.md
View file

@ -1,101 +0,0 @@
---
layout: default
---
<div class="container-fluid">
<div class="row">
<div class="col-md-12">
<div class="page-header">
<h1>
DNSControl: <small>DNS as Code</small>
</h1>
</div>
</div>
</div>
<div class="row">
<div class="col-md-4">
<h2>
Getting Started
</h2>
<p>
Information for new users and the curious.
</p>
<ul>
<li>
<a href="{{site.github.url}}/getting-started">Getting Started</a>: A walk-through of the basics
</li>
<li>
<a href="{{site.github.url}}/provider-list">Providers</a>: Which DNS providers are supported
</li>
<li>
<a href="{{site.github.url}}/examples">Examples</a>: The DNSControl language by example
</li>
<li>
<a href="{{site.github.url}}/migrating">Migrating</a>: Migrating zones to DNSControl
</li>
</ul>
</div>
<div class="col-md-4">
<h2>
Reference
</h2>
<p>
Language resources and procedures.
</p>
<ul>
<li>
<a href="{{site.github.url}}/js">Language Reference</a>: Description of the entire language
</li>
<li>
<a href="{{site.github.url}}/alias">ALIAS / ANAME records in dnscontrol</a>
</li>
<li>
<a href="{{site.github.url}}/spf">SPF Optimizer</a>: Optimize your SPF records
</li>
</ul>
</div>
<div class="col-md-4">
<h2>
Advanced Topics
</h2>
<p>
Take advantage of DNSControl's unique features.
</p>
<ul>
<li>
<a href="">Why CNAME/MX/NS targets require a trailing "dot{{site.github.url}}/why-the-dot"</a>
</li>
<li>
<a href="{{site.github.url}}/unittests">Testing</a>: Unit Testing for you DNS Data
</li>
</ul>
</div>
</div>
<div class="row">
<div class="col-md-12">
<h2>
Developer Info
</h2>
<p>
It is easy to add features and new providers to DNSControl. The code is very modular and easy to modify. There are extensive integration tests that make it easy to boldly make changes with confidence that you'll know if anything is broken. Our mailing list is friendly. Afraid to make your first PR? We'll gladly mentor you through the process. Many major code contributions have come from <a href="https://everythingsysadmin.com/2017/08/go-get-up-to-speed.html">first-time Go users</a>!
</p>
<ul>
<li>
Github: <a href="https://github.com/StackExchange/dnscontrol">https://github.com/StackExchange/dnscontrol</a>
</li>
<li>
Mailing list: <a href="https://groups.google.com/forum/#!forum/dnscontrol-discuss">dnscontrol-discuss</a>: The friendly best place to ask questions and propose new features
</li>
<li>
<a href="{{site.github.url}}/writing-providers">Step-by-Step Guide: Writing Providers</a>: How to write a DNS or Registrar Provider
</li>
<li>
<a href="{{site.github.url}}/adding-new-rtypes">Step-by-Step Guide: Adding new DNS rtypes</a>: How to add a new DNS record type
</li>
</ul>
</div>
</div>
</div>

194
docs/spf.md
View file

@ -1,194 +0,0 @@
---
layout: default
---
# The DNS Control SPF Optimizer
SPF records are hints to email systems that help them determine if
an incoming email message might be spam. The SPF records are placed
in DNS TXT records like so:
$ dig +short google.com txt
"v=spf1 include:_spf.google.com ~all"
SPF records are intentionally limited to 10 verbs that would cause
DNS lookups. In the above example the `include:_spf.google.com`
would cause a DNS lookup. The reason for the "10 lookup limit" is
to make it difficult to leverage the SPF system to create a DDOS
attack on a DNS server.
At StackOverflow, we use many SaaS services and we reached the "10
lookup limit" years ago. We would like to unroll or "inline" the
includes but it would become a maintenance nightmare. What if we
unrolled the SPF include required for Google GSuite and then Google
changed the contents of the SPF records?
We figured that DNSControl could do a better job.
# For the impatient
## Step 1: Define your SPF like this
var SPF_LIST_NORMAL = [
'v=spf1',
'ip4:198.252.206.0/24', // comment
'ip4:192.111.0.0/24', // comment
'include:_spf.google.com', // comment
'include:mailgun.org', // comment
'include:spf-basic.fogcreek.com', // comment
'~all'
].join(" ");
// Change these to the ones that should be flattened:
var SPF_NORMAL = [ // VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV
TXT("@", SPF_LIST_NORMAL, {flatten: "spf-basic.fogcreek.com,mailgun.org", split: "_spf%d"}),
TXT("_rawspf", SPF_LIST_NORMAL) // keep unmodified availible for other tools
]
## Step 2: For a domain that needs that SPF record, include `SPF_NORMAL` as if it is a record.
D('example.com', ...
SPF_NORMAL,
...
)
## Step 3: Push the changes
`dnscontrol preview` and `dnscontrol push` work as you'd expect. However now
your SPF record will be optimized for you.
You might want to check out the web-based SPF tool described below.
## Better comments
Here's how we define our SPF record:
var SPF_SO_LIST = [
'v=spf1',
'ip4:198.252.206.0/24', // ny-mail*
'ip4:192.111.0.0/24', // co-mail*
'include:_spf.google.com', // GSuite
'include:mailgun.org', // Greenhouse.io
'include:spf-basic.fogcreek.com', // Fogbugz
'include:mail.zendesk.com', // Zenddesk
'include:servers.mcsv.net', // MailChimp (Ticket#12345)
'include:sendgrid.net', // SendGrid
'include:spf.mtasv.net', // Desk.com
'~all'
].join(" ");
D('example.com', ...
TXT("@", SPF_SO_LIST),
...
)
The first thing you'll notice is that by defining it this way each
component can include a comment explaining what it is for. This
is important because, and we're not kidding here, for a long time
we didn't know what `include:spf.mtasv.net` was for and we were
afraid to remove it. Finally someone remembered that it was for
Desk.com and we breathed a sigh of relief. You'll also notice that
the Mailchimp entry includes the ticket number of the request to
add it. Now we can refer to that ticket to better understand the
history.
In summary, listing your SPF record like this makes it easier to
maintain a complex SPF record. Certainly you agree that this is
better than `var SPF_SO_LIST = 'v=spf1 'ip4:198.252.206.0/24 'ip4:192.111.0.0/24 'include:_spf.google.com 'include:mailgun.org 'include:spf-basic.fogcreek.com 'include:mail.zendesk.com 'include:servers.mcsv.net 'include:sendgrid.net include:spf.mtasv.net ~all'`
However, we can do better.
# Better macros
Because we don't want to have to remember the "@", and because we
use the same SPF record for multiple domains (any domain that is
attached to our GSuite account), we define a macro called SPF for
use with many domains:
var SPF = [ TXT("@", SPF_SO_LIST) ]
D('example.com', ...
SPF,
...
)
D('otherexample.com', ...
SPF,
...
)
This is a lot less typing. It is also less error-prone: you don't have to remember the `'@'`.
However, we can do better.
# SPF optimizer
As mentioned before, SPF records are intentionally limited to 10
verbs that would cause DNS lookups. This count includes recursive
includes. For example, if you use an `include:` that includes 5
other domains, that's 6 lookups. That leaves you to only 4 more
lookups.
We figured that DNSControl could do better. It could analyze an SPF
record and flatten it to reduce the number of lookups.
However, we're very paranoid. If we break email, a lot of people
notice. Therefore our "flattening" system has some safety rules:
* The system is "opt in". You must specify exactly which includes will be flattened. We recommend you only flatten the minimum needed.
* The flattening works off a cached copy of the DNS lookups. We are concerned
that if someone else's DNS server is down, the optimizer will break and you
won't be able to `dnscontrol push`, which would be very bad especially in
an emergency. Therefore. the process runs off a file called FILLIN but will
warn you if the file needs updating. The updates are easy to do (DNSControl generates
the new file for you to use).
So what does it look like?
Add metadata to the TXT records:
* `flatten: "foo,bar"` (flatten include:foo and include:bar)
* `split: "_spf%d"` (if additional DNS records must be generated, make the labels `_spf1`, `_spf2`, `_spf3`, and so on.)
Here's an example:
var SPF = [
TXT("@", SPF_SO_LIST, {flatten: "spf-basic.fogcreek.com,spf.mtasv.net", split: "_spf%d"}),
TXT("_rawspf", SPF_SO_LIST) // keep unmodified availible for other tools
]
D('example.com', ...
SPF,
...
)
As a result:
* TXT record on `example.com` will be optimized.
* TXT record on `_rawspf.example.com` is the unoptimized version, used purely for demonstration purposes.
You'll notice that we only flatten 2 of all the includes. These are sufficient to get
us to only 10 lookups. They're also the 2 domains that SPF records are the least important.
Thus, if their SPF records change and we don't notice, we won't be too greatly affected.
# Operational Guide
FILL IN THE SEQUENCE OF COMMANDS TO MAINTAIN THE CACHE.
# Interactive mode
To help you decide what to flatten, load `docs/flattener/index.html`
into your web browser and you will be able to play with your SPF
records. We suggest you flatten only the minimum required to reach
10 or fewer lookups.
This tool runs entirely in your browser.
Start interactive mode: [interactive SPF tool](flattener/index.html)
# Future
We'd like to add other optimizations such as:
* De-dup
* Remove overlapping CIDR blocks