Update vendored packages (#326)

* update all vendored packages
This commit is contained in:
Tom Limoncelli 2018-02-27 18:24:11 -05:00 committed by GitHub
parent 54de1ff698
commit 43dc9ac92f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
55 changed files with 12608 additions and 13551 deletions

View file

@ -8,7 +8,6 @@ Have you encountered this situation? Code that has worked for years
suddenly breaks. It turns out someone tried to use it with a file
that came from a MS-Windows system. Now this perfectly good code stops
working.
Looking at a hex dump you realize every other byte is \0. WTF?
No, UTF. More specifically UTF-16LE with an optional BOM.

View file

@ -12,12 +12,13 @@ package utfutil
// utfutil.NewReader() rewraps an existing scanner to make it UTF-encoding agnostic.
// utfutil.BytesReader() takes a []byte and decodes it to UTF-8.
// Since it is impossible to guess 100% correctly if there is no BOM,
// the functions take a 2nd parameter of type "EncodingHint" where you
// specify the default encoding for BOM-less data.
// When there is no BOM, it is impossible to guess correctly 100%
// of the time. Therefore, the functions take a 2nd parameter of type
// "EncodingHint" where you specify the default encoding for BOM-less
// data.
// If someone writes a golang equivalent of uchatdet, I'll add
// a hint called "AUTO" which uses it.
// In the future we'd like to have a hint called AUTO that uses
// uchatdet (or a Go rewrite) to guess.
// Inspiration: I wrote this after spending half a day trying
// to figure out how to use unicode.BOMOverride.

View file

@ -176,7 +176,7 @@ recommend that a file or class name and description of purpose be included on
the same "printed page" as the copyright notice for easier identification within
third-party archives.
Copyright [yyyy] [name of copyright owner]
Copyright 2014 Unknwon
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

15
vendor/github.com/go-ini/ini/Makefile generated vendored Normal file
View file

@ -0,0 +1,15 @@
.PHONY: build test bench vet coverage
build: vet bench
test:
go test -v -cover -race
bench:
go test -v -cover -race -test.bench=. -test.benchmem
vet:
go vet
coverage:
go test -coverprofile=c.out && go tool cover -html=c.out && rm c.out

View file

@ -1,4 +1,4 @@
ini [![Build Status](https://drone.io/github.com/go-ini/ini/status.png)](https://drone.io/github.com/go-ini/ini/latest) [![](http://gocover.io/_badge/github.com/go-ini/ini)](http://gocover.io/github.com/go-ini/ini)
INI [![Build Status](https://travis-ci.org/go-ini/ini.svg?branch=master)](https://travis-ci.org/go-ini/ini) [![Sourcegraph](https://sourcegraph.com/github.com/go-ini/ini/-/badge.svg)](https://sourcegraph.com/github.com/go-ini/ini?badge)
===
![](https://avatars0.githubusercontent.com/u/10216035?v=3&s=200)
@ -9,7 +9,7 @@ Package ini provides INI file read and write functionality in Go.
## Feature
- Load multiple data sources(`[]byte` or file) with overwrites.
- Load multiple data sources(`[]byte`, file and `io.ReadCloser`) with overwrites.
- Read with recursion values.
- Read with parent-child sections.
- Read with auto-increment key names.
@ -22,16 +22,32 @@ Package ini provides INI file read and write functionality in Go.
## Installation
To use a tagged revision:
go get gopkg.in/ini.v1
To use with latest changes:
go get github.com/go-ini/ini
Please add `-u` flag to update in the future.
### Testing
If you want to test on your machine, please apply `-t` flag:
go get -t gopkg.in/ini.v1
Please add `-u` flag to update in the future.
## Getting Started
### Loading from data sources
A **Data Source** is either raw data in type `[]byte` or a file name with type `string` and you can load **as many as** data sources you want. Passing other types will simply return an error.
A **Data Source** is either raw data in type `[]byte`, a file name with type `string` or `io.ReadCloser`. You can load **as many data sources as you want**. Passing other types will simply return an error.
```go
cfg, err := ini.Load([]byte("raw data"), "filename")
cfg, err := ini.Load([]byte("raw data"), "filename", ioutil.NopCloser(bytes.NewReader([]byte("some other data"))))
```
Or start with an empty object:
@ -40,12 +56,78 @@ Or start with an empty object:
cfg := ini.Empty()
```
When you cannot decide how many data sources to load at the beginning, you still able to **Append()** them later.
When you cannot decide how many data sources to load at the beginning, you will still be able to **Append()** them later.
```go
err := cfg.Append("other file", []byte("other raw data"))
```
If you have a list of files with possibilities that some of them may not available at the time, and you don't know exactly which ones, you can use `LooseLoad` to ignore nonexistent files without returning error.
```go
cfg, err := ini.LooseLoad("filename", "filename_404")
```
The cool thing is, whenever the file is available to load while you're calling `Reload` method, it will be counted as usual.
#### Ignore cases of key name
When you do not care about cases of section and key names, you can use `InsensitiveLoad` to force all names to be lowercased while parsing.
```go
cfg, err := ini.InsensitiveLoad("filename")
//...
// sec1 and sec2 are the exactly same section object
sec1, err := cfg.GetSection("Section")
sec2, err := cfg.GetSection("SecTIOn")
// key1 and key2 are the exactly same key object
key1, err := sec1.GetKey("Key")
key2, err := sec2.GetKey("KeY")
```
#### MySQL-like boolean key
MySQL's configuration allows a key without value as follows:
```ini
[mysqld]
...
skip-host-cache
skip-name-resolve
```
By default, this is considered as missing value. But if you know you're going to deal with those cases, you can assign advanced load options:
```go
cfg, err := ini.LoadSources(ini.LoadOptions{AllowBooleanKeys: true}, "my.cnf"))
```
The value of those keys are always `true`, and when you save to a file, it will keep in the same foramt as you read.
To generate such keys in your program, you could use `NewBooleanKey`:
```go
key, err := sec.NewBooleanKey("skip-host-cache")
```
#### Comment
Take care that following format will be treated as comment:
1. Line begins with `#` or `;`
2. Words after `#` or `;`
3. Words after section name (i.e words after `[some section name]`)
If you want to save a value with `#` or `;`, please quote them with ``` ` ``` or ``` """ ```.
Alternatively, you can use following `LoadOptions` to completely ignore inline comments:
```go
cfg, err := ini.LoadSources(ini.LoadOptions{IgnoreInlineComment: true}, "app.ini"))
```
### Working with sections
To get a section, you would need to:
@ -63,7 +145,7 @@ section, err := cfg.GetSection("")
When you're pretty sure the section exists, following code could make your life easier:
```go
section := cfg.Section("")
section := cfg.Section("section name")
```
What happens when the section somehow does not exist? Don't panic, it automatically creates and returns a new section to you.
@ -95,6 +177,12 @@ Same rule applies to key operations:
key := cfg.Section("").Key("key name")
```
To check if a key exists:
```go
yes := cfg.Section("").HasKey("key name")
```
To create a new key:
```go
@ -111,7 +199,7 @@ names := cfg.Section("").KeyStrings()
To get a clone hash of keys and corresponding values:
```go
hash := cfg.GetSection("").KeysHash()
hash := cfg.Section("").KeysHash()
```
### Working with values
@ -133,12 +221,24 @@ val := cfg.Section("").Key("key name").Validate(func(in string) string {
})
```
If you do not want any auto-transformation (such as recursive read) for the values, you can get raw value directly (this way you get much better performance):
```go
val := cfg.Section("").Key("key name").Value()
```
To check if raw value exists:
```go
yes := cfg.Section("").HasValue("test value")
```
To get value with types:
```go
// For boolean values:
// true when value is: 1, t, T, TRUE, true, True, YES, yes, Yes, ON, on, On
// false when value is: 0, f, F, FALSE, false, False, NO, no, No, OFF, off, Off
// true when value is: 1, t, T, TRUE, true, True, YES, yes, Yes, y, ON, on, On
// false when value is: 0, f, F, FALSE, false, False, NO, no, No, n, OFF, off, Off
v, err = cfg.Section("").Key("BOOL").Bool()
v, err = cfg.Section("").Key("FLOAT64").Float64()
v, err = cfg.Section("").Key("INT").Int()
@ -212,6 +312,16 @@ cfg.Section("advance").Key("two_lines").String() // how about continuation lines
cfg.Section("advance").Key("lots_of_lines").String() // 1 2 3 4
```
Well, I hate continuation lines, how do I disable that?
```go
cfg, err := ini.LoadSources(ini.LoadOptions{
IgnoreContinuation: true,
}, "filename")
```
Holy crap!
Note that single quotes around values will be stripped:
```ini
@ -219,6 +329,20 @@ foo = "some value" // foo: some value
bar = 'some value' // bar: some value
```
Sometimes you downloaded file from [Crowdin](https://crowdin.com/) has values like the following (value is surrounded by double quotes and quotes in the value are escaped):
```ini
create_repo="created repository <a href=\"%s\">%s</a>"
```
How do you transform this to regular format automatically?
```go
cfg, err := ini.LoadSources(ini.LoadOptions{UnescapeValueDoubleQuotes: true}, "en-US.ini"))
cfg.Section("<name of your section>").Key("create_repo").String()
// You got: created repository <a href="%s">%s</a>
```
That's all? Hmm, no.
#### Helper methods of working with values
@ -250,9 +374,13 @@ vals = cfg.Section("").Key("TIME").RangeTimeFormat(time.RFC3339, time.Now(), min
vals = cfg.Section("").Key("TIME").RangeTime(time.Now(), minTime, maxTime) // RFC3339
```
To auto-split value into slice:
##### Auto-split values into a slice
To use zero value of type for invalid inputs:
```go
// Input: 1.1, 2.2, 3.3, 4.4 -> [1.1 2.2 3.3 4.4]
// Input: how, 2.2, are, you -> [0.0 2.2 0.0 0.0]
vals = cfg.Section("").Key("STRINGS").Strings(",")
vals = cfg.Section("").Key("FLOAT64S").Float64s(",")
vals = cfg.Section("").Key("INTS").Ints(",")
@ -262,6 +390,32 @@ vals = cfg.Section("").Key("UINT64S").Uint64s(",")
vals = cfg.Section("").Key("TIMES").Times(",")
```
To exclude invalid values out of result slice:
```go
// Input: 1.1, 2.2, 3.3, 4.4 -> [1.1 2.2 3.3 4.4]
// Input: how, 2.2, are, you -> [2.2]
vals = cfg.Section("").Key("FLOAT64S").ValidFloat64s(",")
vals = cfg.Section("").Key("INTS").ValidInts(",")
vals = cfg.Section("").Key("INT64S").ValidInt64s(",")
vals = cfg.Section("").Key("UINTS").ValidUints(",")
vals = cfg.Section("").Key("UINT64S").ValidUint64s(",")
vals = cfg.Section("").Key("TIMES").ValidTimes(",")
```
Or to return nothing but error when have invalid inputs:
```go
// Input: 1.1, 2.2, 3.3, 4.4 -> [1.1 2.2 3.3 4.4]
// Input: how, 2.2, are, you -> error
vals = cfg.Section("").Key("FLOAT64S").StrictFloat64s(",")
vals = cfg.Section("").Key("INTS").StrictInts(",")
vals = cfg.Section("").Key("INT64S").StrictInt64s(",")
vals = cfg.Section("").Key("UINTS").StrictUints(",")
vals = cfg.Section("").Key("UINT64S").StrictUint64s(",")
vals = cfg.Section("").Key("TIMES").StrictTimes(",")
```
### Save your configuration
Finally, it's time to save your configuration to somewhere.
@ -282,6 +436,12 @@ cfg.WriteTo(writer)
cfg.WriteToIndent(writer, "\t")
```
By default, spaces are used to align "=" sign between key and values, to disable that:
```go
ini.PrettyFormat = false
```
## Advanced Usage
### Recursive Values
@ -323,6 +483,27 @@ CLONE_URL = https://%(IMPORT_PATH)s
cfg.Section("package.sub").Key("CLONE_URL").String() // https://gopkg.in/ini.v1
```
#### Retrieve parent keys available to a child section
```go
cfg.Section("package.sub").ParentKeys() // ["CLONE_URL"]
```
### Unparseable Sections
Sometimes, you have sections that do not contain key-value pairs but raw content, to handle such case, you can use `LoadOptions.UnparsableSections`:
```go
cfg, err := ini.LoadSources(ini.LoadOptions{UnparseableSections: []string{"COMMENTS"}}, `[COMMENTS]
<1><L.Slide#2> This slide has the fuel listed in the wrong units <e.1>`))
body := cfg.Section("COMMENTS").Body()
/* --- start ---
<1><L.Slide#2> This slide has the fuel listed in the wrong units <e.1>
------ end --- */
```
### Auto-increment Key Names
If key name is `-` in data source, then it would be seen as special syntax for auto-increment key name start from 1, and every section is independent on counter.
@ -406,18 +587,18 @@ Why not?
```go
type Embeded struct {
Dates []time.Time `delim:"|"`
Places []string
None []int
Dates []time.Time `delim:"|" comment:"Time data"`
Places []string `ini:"places,omitempty"`
None []int `ini:",omitempty"`
}
type Author struct {
Name string `ini:"NAME"`
Male bool
Age int
Age int `comment:"Author's age"`
GPA float64
NeverMind string `ini:"-"`
*Embeded
*Embeded `comment:"Embeded section"`
}
func main() {
@ -438,13 +619,15 @@ So, what do I get?
```ini
NAME = Unknwon
Male = true
; Author's age
Age = 21
GPA = 2.8
; Embeded section
[Embeded]
; Time data
Dates = 2015-08-07T22:14:22+08:00|2015-08-07T22:14:22+08:00
Places = HangZhou,Boston
None =
places = HangZhou,Boston
```
#### Name Mapper
@ -464,7 +647,7 @@ type Info struct {
}
func main() {
err = ini.MapToWithMapper(&Info{}, ini.TitleUnderscore, []byte("packag_name=ini"))
err = ini.MapToWithMapper(&Info{}, ini.TitleUnderscore, []byte("package_name=ini"))
// ...
cfg, err := ini.Load([]byte("PACKAGE_NAME=ini"))
@ -478,6 +661,26 @@ func main() {
Same rules of name mapper apply to `ini.ReflectFromWithMapper` function.
#### Value Mapper
To expand values (e.g. from environment variables), you can use the `ValueMapper` to transform values:
```go
type Env struct {
Foo string `ini:"foo"`
}
func main() {
cfg, err := ini.Load([]byte("[env]\nfoo = ${MY_VAR}\n")
cfg.ValueMapper = os.ExpandEnv
// ...
env := &Env{}
err = cfg.Section("env").MapTo(env)
}
```
This would set the value of `env.Foo` to the value of the environment variable `MY_VAR`.
#### Other Notes On Map/Reflect
Any embedded struct is treated as a section by default, and there is no automatic parent-child relations in map/reflect feature:

View file

@ -2,7 +2,7 @@
## 功能特性
- 支持覆盖加载多个数据源(`[]byte` 或文件
- 支持覆盖加载多个数据源(`[]byte`、文件和 `io.ReadCloser`
- 支持递归读取键值
- 支持读取父子分区
- 支持读取自增键名
@ -15,16 +15,32 @@
## 下载安装
使用一个特定版本:
go get gopkg.in/ini.v1
使用最新版:
go get github.com/go-ini/ini
如需更新请添加 `-u` 选项。
### 测试安装
如果您想要在自己的机器上运行测试,请使用 `-t` 标记:
go get -t gopkg.in/ini.v1
如需更新请添加 `-u` 选项。
## 开始使用
### 从数据源加载
一个 **数据源** 可以是 `[]byte` 类型的原始数据,或 `string` 类型的文件路径。您可以加载 **任意多个** 数据源。如果您传递其它类型的数据源,则会直接返回错误。
一个 **数据源** 可以是 `[]byte` 类型的原始数据,`string` 类型的文件路径`io.ReadCloser`。您可以加载 **任意多个** 数据源。如果您传递其它类型的数据源,则会直接返回错误。
```go
cfg, err := ini.Load([]byte("raw data"), "filename")
cfg, err := ini.Load([]byte("raw data"), "filename", ioutil.NopCloser(bytes.NewReader([]byte("some other data"))))
```
或者从一个空白的文件开始:
@ -39,6 +55,72 @@ cfg := ini.Empty()
err := cfg.Append("other file", []byte("other raw data"))
```
当您想要加载一系列文件,但是不能够确定其中哪些文件是不存在的,可以通过调用函数 `LooseLoad` 来忽略它们(`Load` 会因为文件不存在而返回错误):
```go
cfg, err := ini.LooseLoad("filename", "filename_404")
```
更牛逼的是,当那些之前不存在的文件在重新调用 `Reload` 方法的时候突然出现了,那么它们会被正常加载。
#### 忽略键名的大小写
有时候分区和键的名称大小写混合非常烦人,这个时候就可以通过 `InsensitiveLoad` 将所有分区和键名在读取里强制转换为小写:
```go
cfg, err := ini.InsensitiveLoad("filename")
//...
// sec1 和 sec2 指向同一个分区对象
sec1, err := cfg.GetSection("Section")
sec2, err := cfg.GetSection("SecTIOn")
// key1 和 key2 指向同一个键对象
key1, err := sec1.GetKey("Key")
key2, err := sec2.GetKey("KeY")
```
#### 类似 MySQL 配置中的布尔值键
MySQL 的配置文件中会出现没有具体值的布尔类型的键:
```ini
[mysqld]
...
skip-host-cache
skip-name-resolve
```
默认情况下这被认为是缺失值而无法完成解析,但可以通过高级的加载选项对它们进行处理:
```go
cfg, err := ini.LoadSources(ini.LoadOptions{AllowBooleanKeys: true}, "my.cnf"))
```
这些键的值永远为 `true`,且在保存到文件时也只会输出键名。
如果您想要通过程序来生成此类键,则可以使用 `NewBooleanKey`
```go
key, err := sec.NewBooleanKey("skip-host-cache")
```
#### 关于注释
下述几种情况的内容将被视为注释:
1. 所有以 `#``;` 开头的行
2. 所有在 `#``;` 之后的内容
3. 分区标签后的文字 (即 `[分区名]` 之后的内容)
如果你希望使用包含 `#``;` 的值,请使用 ``` ` ``` 或 ``` """ ``` 进行包覆。
除此之外,您还可以通过 `LoadOptions` 完全忽略行内注释:
```go
cfg, err := ini.LoadSources(ini.LoadOptions{IgnoreInlineComment: true}, "app.ini"))
```
### 操作分区Section
获取指定分区:
@ -56,7 +138,7 @@ section, err := cfg.GetSection("")
当您非常确定某个分区是存在的,可以使用以下简便方法:
```go
section := cfg.Section("")
section := cfg.Section("section name")
```
如果不小心判断错了,要获取的分区其实是不存在的,那会发生什么呢?没事的,它会自动创建并返回一个对应的分区对象给您。
@ -88,6 +170,12 @@ key, err := cfg.Section("").GetKey("key name")
key := cfg.Section("").Key("key name")
```
判断某个键是否存在:
```go
yes := cfg.Section("").HasKey("key name")
```
创建一个新的键:
```go
@ -104,7 +192,7 @@ names := cfg.Section("").KeyStrings()
获取分区下的所有键值对的克隆:
```go
hash := cfg.GetSection("").KeysHash()
hash := cfg.Section("").KeysHash()
```
### 操作键值Value
@ -126,12 +214,24 @@ val := cfg.Section("").Key("key name").Validate(func(in string) string {
})
```
如果您不需要任何对值的自动转变功能(例如递归读取),可以直接获取原值(这种方式性能最佳):
```go
val := cfg.Section("").Key("key name").Value()
```
判断某个原值是否存在:
```go
yes := cfg.Section("").HasValue("test value")
```
获取其它类型的值:
```go
// 布尔值的规则:
// true 当值为1, t, T, TRUE, true, True, YES, yes, Yes, ON, on, On
// false 当值为0, f, F, FALSE, false, False, NO, no, No, OFF, off, Off
// true 当值为1, t, T, TRUE, true, True, YES, yes, Yes, y, ON, on, On
// false 当值为0, f, F, FALSE, false, False, NO, no, No, n, OFF, off, Off
v, err = cfg.Section("").Key("BOOL").Bool()
v, err = cfg.Section("").Key("FLOAT64").Float64()
v, err = cfg.Section("").Key("INT").Int()
@ -205,6 +305,16 @@ cfg.Section("advance").Key("two_lines").String() // how about continuation lines
cfg.Section("advance").Key("lots_of_lines").String() // 1 2 3 4
```
可是我有时候觉得两行连在一起特别没劲,怎么才能不自动连接两行呢?
```go
cfg, err := ini.LoadSources(ini.LoadOptions{
IgnoreContinuation: true,
}, "filename")
```
哇靠给力啊!
需要注意的是,值两侧的单引号会被自动剔除:
```ini
@ -212,6 +322,20 @@ foo = "some value" // foo: some value
bar = 'some value' // bar: some value
```
有时您会获得像从 [Crowdin](https://crowdin.com/) 网站下载的文件那样具有特殊格式的值(值使用双引号括起来,内部的双引号被转义):
```ini
create_repo="创建了仓库 <a href=\"%s\">%s</a>"
```
那么,怎么自动地将这类值进行处理呢?
```go
cfg, err := ini.LoadSources(ini.LoadOptions{UnescapeValueDoubleQuotes: true}, "en-US.ini"))
cfg.Section("<name of your section>").Key("create_repo").String()
// You got: 创建了仓库 <a href="%s">%s</a>
```
这就是全部了?哈哈,当然不是。
#### 操作键值的辅助方法
@ -243,9 +367,13 @@ vals = cfg.Section("").Key("TIME").RangeTimeFormat(time.RFC3339, time.Now(), min
vals = cfg.Section("").Key("TIME").RangeTime(time.Now(), minTime, maxTime) // RFC3339
```
自动分割键值为切片slice
##### 自动分割键值到切片slice
当存在无效输入时,使用零值代替:
```go
// Input: 1.1, 2.2, 3.3, 4.4 -> [1.1 2.2 3.3 4.4]
// Input: how, 2.2, are, you -> [0.0 2.2 0.0 0.0]
vals = cfg.Section("").Key("STRINGS").Strings(",")
vals = cfg.Section("").Key("FLOAT64S").Float64s(",")
vals = cfg.Section("").Key("INTS").Ints(",")
@ -255,6 +383,32 @@ vals = cfg.Section("").Key("UINT64S").Uint64s(",")
vals = cfg.Section("").Key("TIMES").Times(",")
```
从结果切片中剔除无效输入:
```go
// Input: 1.1, 2.2, 3.3, 4.4 -> [1.1 2.2 3.3 4.4]
// Input: how, 2.2, are, you -> [2.2]
vals = cfg.Section("").Key("FLOAT64S").ValidFloat64s(",")
vals = cfg.Section("").Key("INTS").ValidInts(",")
vals = cfg.Section("").Key("INT64S").ValidInt64s(",")
vals = cfg.Section("").Key("UINTS").ValidUints(",")
vals = cfg.Section("").Key("UINT64S").ValidUint64s(",")
vals = cfg.Section("").Key("TIMES").ValidTimes(",")
```
当存在无效输入时,直接返回错误:
```go
// Input: 1.1, 2.2, 3.3, 4.4 -> [1.1 2.2 3.3 4.4]
// Input: how, 2.2, are, you -> error
vals = cfg.Section("").Key("FLOAT64S").StrictFloat64s(",")
vals = cfg.Section("").Key("INTS").StrictInts(",")
vals = cfg.Section("").Key("INT64S").StrictInt64s(",")
vals = cfg.Section("").Key("UINTS").StrictUints(",")
vals = cfg.Section("").Key("UINT64S").StrictUint64s(",")
vals = cfg.Section("").Key("TIMES").StrictTimes(",")
```
### 保存配置
终于到了这个时刻,是时候保存一下配置了。
@ -275,9 +429,15 @@ cfg.WriteTo(writer)
cfg.WriteToIndent(writer, "\t")
```
### 高级用法
默认情况下,空格将被用于对齐键值之间的等号以美化输出结果,以下代码可以禁用该功能:
#### 递归读取键值
```go
ini.PrettyFormat = false
```
## 高级用法
### 递归读取键值
在获取所有键值的过程中,特殊语法 `%(<name>)s` 会被应用,其中 `<name>` 可以是相同分区或者默认分区下的键名。字符串 `%(<name>)s` 会被相应的键值所替代,如果指定的键不存在,则会用空字符串替代。您可以最多使用 99 层的递归嵌套。
@ -297,7 +457,7 @@ cfg.Section("author").Key("GITHUB").String() // https://github.com/Unknwon
cfg.Section("package").Key("FULL_NAME").String() // github.com/go-ini/ini
```
#### 读取父子分区
### 读取父子分区
您可以在分区名称中使用 `.` 来表示两个或多个分区之间的父子关系。如果某个键在子分区中不存在,则会去它的父分区中再次寻找,直到没有父分区为止。
@ -316,7 +476,28 @@ CLONE_URL = https://%(IMPORT_PATH)s
cfg.Section("package.sub").Key("CLONE_URL").String() // https://gopkg.in/ini.v1
```
#### 读取自增键名
#### 获取上级父分区下的所有键名
```go
cfg.Section("package.sub").ParentKeys() // ["CLONE_URL"]
```
### 无法解析的分区
如果遇到一些比较特殊的分区,它们不包含常见的键值对,而是没有固定格式的纯文本,则可以使用 `LoadOptions.UnparsableSections` 进行处理:
```go
cfg, err := LoadSources(ini.LoadOptions{UnparseableSections: []string{"COMMENTS"}}, `[COMMENTS]
<1><L.Slide#2> This slide has the fuel listed in the wrong units <e.1>`))
body := cfg.Section("COMMENTS").Body()
/* --- start ---
<1><L.Slide#2> This slide has the fuel listed in the wrong units <e.1>
------ end --- */
```
### 读取自增键名
如果数据源中的键名为 `-`,则认为该键使用了自增键名的特殊语法。计数器从 1 开始,并且分区之间是相互独立的。
@ -397,18 +578,18 @@ p := &Person{
```go
type Embeded struct {
Dates []time.Time `delim:"|"`
Places []string
None []int
Dates []time.Time `delim:"|" comment:"Time data"`
Places []string `ini:"places,omitempty"`
None []int `ini:",omitempty"`
}
type Author struct {
Name string `ini:"NAME"`
Male bool
Age int
Age int `comment:"Author's age"`
GPA float64
NeverMind string `ini:"-"`
*Embeded
*Embeded `comment:"Embeded section"`
}
func main() {
@ -429,13 +610,15 @@ func main() {
```ini
NAME = Unknwon
Male = true
; Author's age
Age = 21
GPA = 2.8
; Embeded section
[Embeded]
; Time data
Dates = 2015-08-07T22:14:22+08:00|2015-08-07T22:14:22+08:00
Places = HangZhou,Boston
None =
places = HangZhou,Boston
```
#### 名称映射器Name Mapper
@ -455,7 +638,7 @@ type Info struct{
}
func main() {
err = ini.MapToWithMapper(&Info{}, ini.TitleUnderscore, []byte("packag_name=ini"))
err = ini.MapToWithMapper(&Info{}, ini.TitleUnderscore, []byte("package_name=ini"))
// ...
cfg, err := ini.Load([]byte("PACKAGE_NAME=ini"))
@ -469,6 +652,26 @@ func main() {
使用函数 `ini.ReflectFromWithMapper` 时也可应用相同的规则。
#### 值映射器Value Mapper
值映射器允许使用一个自定义函数自动展开值的具体内容,例如:运行时获取环境变量:
```go
type Env struct {
Foo string `ini:"foo"`
}
func main() {
cfg, err := ini.Load([]byte("[env]\nfoo = ${MY_VAR}\n")
cfg.ValueMapper = os.ExpandEnv
// ...
env := &Env{}
err = cfg.Section("env").MapTo(env)
}
```
本例中,`env.Foo` 将会是运行时所获取到环境变量 `MY_VAR` 的值。
#### 映射/反射的其它说明
任何嵌入的结构都会被默认认作一个不同的分区,并且不会自动产生所谓的父子分区关联:

32
vendor/github.com/go-ini/ini/error.go generated vendored Normal file
View file

@ -0,0 +1,32 @@
// Copyright 2016 Unknwon
//
// Licensed under the Apache License, Version 2.0 (the "License"): you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations
// under the License.
package ini
import (
"fmt"
)
type ErrDelimiterNotFound struct {
Line string
}
func IsErrDelimiterNotFound(err error) bool {
_, ok := err.(ErrDelimiterNotFound)
return ok
}
func (err ErrDelimiterNotFound) Error() string {
return fmt.Sprintf("key-value delimiter not found: %s", err.Line)
}

398
vendor/github.com/go-ini/ini/file.go generated vendored Normal file
View file

@ -0,0 +1,398 @@
// Copyright 2017 Unknwon
//
// Licensed under the Apache License, Version 2.0 (the "License"): you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations
// under the License.
package ini
import (
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"strings"
"sync"
)
// File represents a combination of a or more INI file(s) in memory.
type File struct {
options LoadOptions
dataSources []dataSource
// Should make things safe, but sometimes doesn't matter.
BlockMode bool
lock sync.RWMutex
// To keep data in order.
sectionList []string
// Actual data is stored here.
sections map[string]*Section
NameMapper
ValueMapper
}
// newFile initializes File object with given data sources.
func newFile(dataSources []dataSource, opts LoadOptions) *File {
return &File{
BlockMode: true,
dataSources: dataSources,
sections: make(map[string]*Section),
sectionList: make([]string, 0, 10),
options: opts,
}
}
// Empty returns an empty file object.
func Empty() *File {
// Ignore error here, we sure our data is good.
f, _ := Load([]byte(""))
return f
}
// NewSection creates a new section.
func (f *File) NewSection(name string) (*Section, error) {
if len(name) == 0 {
return nil, errors.New("error creating new section: empty section name")
} else if f.options.Insensitive && name != DEFAULT_SECTION {
name = strings.ToLower(name)
}
if f.BlockMode {
f.lock.Lock()
defer f.lock.Unlock()
}
if inSlice(name, f.sectionList) {
return f.sections[name], nil
}
f.sectionList = append(f.sectionList, name)
f.sections[name] = newSection(f, name)
return f.sections[name], nil
}
// NewRawSection creates a new section with an unparseable body.
func (f *File) NewRawSection(name, body string) (*Section, error) {
section, err := f.NewSection(name)
if err != nil {
return nil, err
}
section.isRawSection = true
section.rawBody = body
return section, nil
}
// NewSections creates a list of sections.
func (f *File) NewSections(names ...string) (err error) {
for _, name := range names {
if _, err = f.NewSection(name); err != nil {
return err
}
}
return nil
}
// GetSection returns section by given name.
func (f *File) GetSection(name string) (*Section, error) {
if len(name) == 0 {
name = DEFAULT_SECTION
}
if f.options.Insensitive {
name = strings.ToLower(name)
}
if f.BlockMode {
f.lock.RLock()
defer f.lock.RUnlock()
}
sec := f.sections[name]
if sec == nil {
return nil, fmt.Errorf("section '%s' does not exist", name)
}
return sec, nil
}
// Section assumes named section exists and returns a zero-value when not.
func (f *File) Section(name string) *Section {
sec, err := f.GetSection(name)
if err != nil {
// Note: It's OK here because the only possible error is empty section name,
// but if it's empty, this piece of code won't be executed.
sec, _ = f.NewSection(name)
return sec
}
return sec
}
// Section returns list of Section.
func (f *File) Sections() []*Section {
sections := make([]*Section, len(f.sectionList))
for i := range f.sectionList {
sections[i] = f.Section(f.sectionList[i])
}
return sections
}
// ChildSections returns a list of child sections of given section name.
func (f *File) ChildSections(name string) []*Section {
return f.Section(name).ChildSections()
}
// SectionStrings returns list of section names.
func (f *File) SectionStrings() []string {
list := make([]string, len(f.sectionList))
copy(list, f.sectionList)
return list
}
// DeleteSection deletes a section.
func (f *File) DeleteSection(name string) {
if f.BlockMode {
f.lock.Lock()
defer f.lock.Unlock()
}
if len(name) == 0 {
name = DEFAULT_SECTION
}
for i, s := range f.sectionList {
if s == name {
f.sectionList = append(f.sectionList[:i], f.sectionList[i+1:]...)
delete(f.sections, name)
return
}
}
}
func (f *File) reload(s dataSource) error {
r, err := s.ReadCloser()
if err != nil {
return err
}
defer r.Close()
return f.parse(r)
}
// Reload reloads and parses all data sources.
func (f *File) Reload() (err error) {
for _, s := range f.dataSources {
if err = f.reload(s); err != nil {
// In loose mode, we create an empty default section for nonexistent files.
if os.IsNotExist(err) && f.options.Loose {
f.parse(bytes.NewBuffer(nil))
continue
}
return err
}
}
return nil
}
// Append appends one or more data sources and reloads automatically.
func (f *File) Append(source interface{}, others ...interface{}) error {
ds, err := parseDataSource(source)
if err != nil {
return err
}
f.dataSources = append(f.dataSources, ds)
for _, s := range others {
ds, err = parseDataSource(s)
if err != nil {
return err
}
f.dataSources = append(f.dataSources, ds)
}
return f.Reload()
}
func (f *File) writeToBuffer(indent string) (*bytes.Buffer, error) {
equalSign := "="
if PrettyFormat {
equalSign = " = "
}
// Use buffer to make sure target is safe until finish encoding.
buf := bytes.NewBuffer(nil)
for i, sname := range f.sectionList {
sec := f.Section(sname)
if len(sec.Comment) > 0 {
if sec.Comment[0] != '#' && sec.Comment[0] != ';' {
sec.Comment = "; " + sec.Comment
} else {
sec.Comment = sec.Comment[:1] + " " + strings.TrimSpace(sec.Comment[1:])
}
if _, err := buf.WriteString(sec.Comment + LineBreak); err != nil {
return nil, err
}
}
if i > 0 || DefaultHeader {
if _, err := buf.WriteString("[" + sname + "]" + LineBreak); err != nil {
return nil, err
}
} else {
// Write nothing if default section is empty
if len(sec.keyList) == 0 {
continue
}
}
if sec.isRawSection {
if _, err := buf.WriteString(sec.rawBody); err != nil {
return nil, err
}
if PrettySection {
// Put a line between sections
if _, err := buf.WriteString(LineBreak); err != nil {
return nil, err
}
}
continue
}
// Count and generate alignment length and buffer spaces using the
// longest key. Keys may be modifed if they contain certain characters so
// we need to take that into account in our calculation.
alignLength := 0
if PrettyFormat {
for _, kname := range sec.keyList {
keyLength := len(kname)
// First case will surround key by ` and second by """
if strings.ContainsAny(kname, "\"=:") {
keyLength += 2
} else if strings.Contains(kname, "`") {
keyLength += 6
}
if keyLength > alignLength {
alignLength = keyLength
}
}
}
alignSpaces := bytes.Repeat([]byte(" "), alignLength)
KEY_LIST:
for _, kname := range sec.keyList {
key := sec.Key(kname)
if len(key.Comment) > 0 {
if len(indent) > 0 && sname != DEFAULT_SECTION {
buf.WriteString(indent)
}
if key.Comment[0] != '#' && key.Comment[0] != ';' {
key.Comment = "; " + key.Comment
} else {
key.Comment = key.Comment[:1] + " " + strings.TrimSpace(key.Comment[1:])
}
if _, err := buf.WriteString(key.Comment + LineBreak); err != nil {
return nil, err
}
}
if len(indent) > 0 && sname != DEFAULT_SECTION {
buf.WriteString(indent)
}
switch {
case key.isAutoIncrement:
kname = "-"
case strings.ContainsAny(kname, "\"=:"):
kname = "`" + kname + "`"
case strings.Contains(kname, "`"):
kname = `"""` + kname + `"""`
}
for _, val := range key.ValueWithShadows() {
if _, err := buf.WriteString(kname); err != nil {
return nil, err
}
if key.isBooleanType {
if kname != sec.keyList[len(sec.keyList)-1] {
buf.WriteString(LineBreak)
}
continue KEY_LIST
}
// Write out alignment spaces before "=" sign
if PrettyFormat {
buf.Write(alignSpaces[:alignLength-len(kname)])
}
// In case key value contains "\n", "`", "\"", "#" or ";"
if strings.ContainsAny(val, "\n`") {
val = `"""` + val + `"""`
} else if !f.options.IgnoreInlineComment && strings.ContainsAny(val, "#;") {
val = "`" + val + "`"
}
if _, err := buf.WriteString(equalSign + val + LineBreak); err != nil {
return nil, err
}
}
for _, val := range key.nestedValues {
if _, err := buf.WriteString(indent + " " + val + LineBreak); err != nil {
return nil, err
}
}
}
if PrettySection {
// Put a line between sections
if _, err := buf.WriteString(LineBreak); err != nil {
return nil, err
}
}
}
return buf, nil
}
// WriteToIndent writes content into io.Writer with given indention.
// If PrettyFormat has been set to be true,
// it will align "=" sign with spaces under each section.
func (f *File) WriteToIndent(w io.Writer, indent string) (int64, error) {
buf, err := f.writeToBuffer(indent)
if err != nil {
return 0, err
}
return buf.WriteTo(w)
}
// WriteTo writes file content into io.Writer.
func (f *File) WriteTo(w io.Writer) (int64, error) {
return f.WriteToIndent(w, "")
}
// SaveToIndent writes content to file system with given value indention.
func (f *File) SaveToIndent(filename, indent string) error {
// Note: Because we are truncating with os.Create,
// so it's safer to save to a temporary file location and rename afte done.
buf, err := f.writeToBuffer(indent)
if err != nil {
return err
}
return ioutil.WriteFile(filename, buf.Bytes(), 0666)
}
// SaveTo writes content to file system.
func (f *File) SaveTo(filename string) error {
return f.SaveToIndent(filename, "")
}

1192
vendor/github.com/go-ini/ini/ini.go generated vendored

File diff suppressed because it is too large Load diff

751
vendor/github.com/go-ini/ini/key.go generated vendored Normal file
View file

@ -0,0 +1,751 @@
// Copyright 2014 Unknwon
//
// Licensed under the Apache License, Version 2.0 (the "License"): you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations
// under the License.
package ini
import (
"bytes"
"errors"
"fmt"
"strconv"
"strings"
"time"
)
// Key represents a key under a section.
type Key struct {
s *Section
Comment string
name string
value string
isAutoIncrement bool
isBooleanType bool
isShadow bool
shadows []*Key
nestedValues []string
}
// newKey simply return a key object with given values.
func newKey(s *Section, name, val string) *Key {
return &Key{
s: s,
name: name,
value: val,
}
}
func (k *Key) addShadow(val string) error {
if k.isShadow {
return errors.New("cannot add shadow to another shadow key")
} else if k.isAutoIncrement || k.isBooleanType {
return errors.New("cannot add shadow to auto-increment or boolean key")
}
shadow := newKey(k.s, k.name, val)
shadow.isShadow = true
k.shadows = append(k.shadows, shadow)
return nil
}
// AddShadow adds a new shadow key to itself.
func (k *Key) AddShadow(val string) error {
if !k.s.f.options.AllowShadows {
return errors.New("shadow key is not allowed")
}
return k.addShadow(val)
}
func (k *Key) addNestedValue(val string) error {
if k.isAutoIncrement || k.isBooleanType {
return errors.New("cannot add nested value to auto-increment or boolean key")
}
k.nestedValues = append(k.nestedValues, val)
return nil
}
func (k *Key) AddNestedValue(val string) error {
if !k.s.f.options.AllowNestedValues {
return errors.New("nested value is not allowed")
}
return k.addNestedValue(val)
}
// ValueMapper represents a mapping function for values, e.g. os.ExpandEnv
type ValueMapper func(string) string
// Name returns name of key.
func (k *Key) Name() string {
return k.name
}
// Value returns raw value of key for performance purpose.
func (k *Key) Value() string {
return k.value
}
// ValueWithShadows returns raw values of key and its shadows if any.
func (k *Key) ValueWithShadows() []string {
if len(k.shadows) == 0 {
return []string{k.value}
}
vals := make([]string, len(k.shadows)+1)
vals[0] = k.value
for i := range k.shadows {
vals[i+1] = k.shadows[i].value
}
return vals
}
// NestedValues returns nested values stored in the key.
// It is possible returned value is nil if no nested values stored in the key.
func (k *Key) NestedValues() []string {
return k.nestedValues
}
// transformValue takes a raw value and transforms to its final string.
func (k *Key) transformValue(val string) string {
if k.s.f.ValueMapper != nil {
val = k.s.f.ValueMapper(val)
}
// Fail-fast if no indicate char found for recursive value
if !strings.Contains(val, "%") {
return val
}
for i := 0; i < _DEPTH_VALUES; i++ {
vr := varPattern.FindString(val)
if len(vr) == 0 {
break
}
// Take off leading '%(' and trailing ')s'.
noption := strings.TrimLeft(vr, "%(")
noption = strings.TrimRight(noption, ")s")
// Search in the same section.
nk, err := k.s.GetKey(noption)
if err != nil || k == nk {
// Search again in default section.
nk, _ = k.s.f.Section("").GetKey(noption)
}
// Substitute by new value and take off leading '%(' and trailing ')s'.
val = strings.Replace(val, vr, nk.value, -1)
}
return val
}
// String returns string representation of value.
func (k *Key) String() string {
return k.transformValue(k.value)
}
// Validate accepts a validate function which can
// return modifed result as key value.
func (k *Key) Validate(fn func(string) string) string {
return fn(k.String())
}
// parseBool returns the boolean value represented by the string.
//
// It accepts 1, t, T, TRUE, true, True, YES, yes, Yes, y, ON, on, On,
// 0, f, F, FALSE, false, False, NO, no, No, n, OFF, off, Off.
// Any other value returns an error.
func parseBool(str string) (value bool, err error) {
switch str {
case "1", "t", "T", "true", "TRUE", "True", "YES", "yes", "Yes", "y", "ON", "on", "On":
return true, nil
case "0", "f", "F", "false", "FALSE", "False", "NO", "no", "No", "n", "OFF", "off", "Off":
return false, nil
}
return false, fmt.Errorf("parsing \"%s\": invalid syntax", str)
}
// Bool returns bool type value.
func (k *Key) Bool() (bool, error) {
return parseBool(k.String())
}
// Float64 returns float64 type value.
func (k *Key) Float64() (float64, error) {
return strconv.ParseFloat(k.String(), 64)
}
// Int returns int type value.
func (k *Key) Int() (int, error) {
return strconv.Atoi(k.String())
}
// Int64 returns int64 type value.
func (k *Key) Int64() (int64, error) {
return strconv.ParseInt(k.String(), 10, 64)
}
// Uint returns uint type valued.
func (k *Key) Uint() (uint, error) {
u, e := strconv.ParseUint(k.String(), 10, 64)
return uint(u), e
}
// Uint64 returns uint64 type value.
func (k *Key) Uint64() (uint64, error) {
return strconv.ParseUint(k.String(), 10, 64)
}
// Duration returns time.Duration type value.
func (k *Key) Duration() (time.Duration, error) {
return time.ParseDuration(k.String())
}
// TimeFormat parses with given format and returns time.Time type value.
func (k *Key) TimeFormat(format string) (time.Time, error) {
return time.Parse(format, k.String())
}
// Time parses with RFC3339 format and returns time.Time type value.
func (k *Key) Time() (time.Time, error) {
return k.TimeFormat(time.RFC3339)
}
// MustString returns default value if key value is empty.
func (k *Key) MustString(defaultVal string) string {
val := k.String()
if len(val) == 0 {
k.value = defaultVal
return defaultVal
}
return val
}
// MustBool always returns value without error,
// it returns false if error occurs.
func (k *Key) MustBool(defaultVal ...bool) bool {
val, err := k.Bool()
if len(defaultVal) > 0 && err != nil {
k.value = strconv.FormatBool(defaultVal[0])
return defaultVal[0]
}
return val
}
// MustFloat64 always returns value without error,
// it returns 0.0 if error occurs.
func (k *Key) MustFloat64(defaultVal ...float64) float64 {
val, err := k.Float64()
if len(defaultVal) > 0 && err != nil {
k.value = strconv.FormatFloat(defaultVal[0], 'f', -1, 64)
return defaultVal[0]
}
return val
}
// MustInt always returns value without error,
// it returns 0 if error occurs.
func (k *Key) MustInt(defaultVal ...int) int {
val, err := k.Int()
if len(defaultVal) > 0 && err != nil {
k.value = strconv.FormatInt(int64(defaultVal[0]), 10)
return defaultVal[0]
}
return val
}
// MustInt64 always returns value without error,
// it returns 0 if error occurs.
func (k *Key) MustInt64(defaultVal ...int64) int64 {
val, err := k.Int64()
if len(defaultVal) > 0 && err != nil {
k.value = strconv.FormatInt(defaultVal[0], 10)
return defaultVal[0]
}
return val
}
// MustUint always returns value without error,
// it returns 0 if error occurs.
func (k *Key) MustUint(defaultVal ...uint) uint {
val, err := k.Uint()
if len(defaultVal) > 0 && err != nil {
k.value = strconv.FormatUint(uint64(defaultVal[0]), 10)
return defaultVal[0]
}
return val
}
// MustUint64 always returns value without error,
// it returns 0 if error occurs.
func (k *Key) MustUint64(defaultVal ...uint64) uint64 {
val, err := k.Uint64()
if len(defaultVal) > 0 && err != nil {
k.value = strconv.FormatUint(defaultVal[0], 10)
return defaultVal[0]
}
return val
}
// MustDuration always returns value without error,
// it returns zero value if error occurs.
func (k *Key) MustDuration(defaultVal ...time.Duration) time.Duration {
val, err := k.Duration()
if len(defaultVal) > 0 && err != nil {
k.value = defaultVal[0].String()
return defaultVal[0]
}
return val
}
// MustTimeFormat always parses with given format and returns value without error,
// it returns zero value if error occurs.
func (k *Key) MustTimeFormat(format string, defaultVal ...time.Time) time.Time {
val, err := k.TimeFormat(format)
if len(defaultVal) > 0 && err != nil {
k.value = defaultVal[0].Format(format)
return defaultVal[0]
}
return val
}
// MustTime always parses with RFC3339 format and returns value without error,
// it returns zero value if error occurs.
func (k *Key) MustTime(defaultVal ...time.Time) time.Time {
return k.MustTimeFormat(time.RFC3339, defaultVal...)
}
// In always returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) In(defaultVal string, candidates []string) string {
val := k.String()
for _, cand := range candidates {
if val == cand {
return val
}
}
return defaultVal
}
// InFloat64 always returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) InFloat64(defaultVal float64, candidates []float64) float64 {
val := k.MustFloat64()
for _, cand := range candidates {
if val == cand {
return val
}
}
return defaultVal
}
// InInt always returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) InInt(defaultVal int, candidates []int) int {
val := k.MustInt()
for _, cand := range candidates {
if val == cand {
return val
}
}
return defaultVal
}
// InInt64 always returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) InInt64(defaultVal int64, candidates []int64) int64 {
val := k.MustInt64()
for _, cand := range candidates {
if val == cand {
return val
}
}
return defaultVal
}
// InUint always returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) InUint(defaultVal uint, candidates []uint) uint {
val := k.MustUint()
for _, cand := range candidates {
if val == cand {
return val
}
}
return defaultVal
}
// InUint64 always returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) InUint64(defaultVal uint64, candidates []uint64) uint64 {
val := k.MustUint64()
for _, cand := range candidates {
if val == cand {
return val
}
}
return defaultVal
}
// InTimeFormat always parses with given format and returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) InTimeFormat(format string, defaultVal time.Time, candidates []time.Time) time.Time {
val := k.MustTimeFormat(format)
for _, cand := range candidates {
if val == cand {
return val
}
}
return defaultVal
}
// InTime always parses with RFC3339 format and returns value without error,
// it returns default value if error occurs or doesn't fit into candidates.
func (k *Key) InTime(defaultVal time.Time, candidates []time.Time) time.Time {
return k.InTimeFormat(time.RFC3339, defaultVal, candidates)
}
// RangeFloat64 checks if value is in given range inclusively,
// and returns default value if it's not.
func (k *Key) RangeFloat64(defaultVal, min, max float64) float64 {
val := k.MustFloat64()
if val < min || val > max {
return defaultVal
}
return val
}
// RangeInt checks if value is in given range inclusively,
// and returns default value if it's not.
func (k *Key) RangeInt(defaultVal, min, max int) int {
val := k.MustInt()
if val < min || val > max {
return defaultVal
}
return val
}
// RangeInt64 checks if value is in given range inclusively,
// and returns default value if it's not.
func (k *Key) RangeInt64(defaultVal, min, max int64) int64 {
val := k.MustInt64()
if val < min || val > max {
return defaultVal
}
return val
}
// RangeTimeFormat checks if value with given format is in given range inclusively,
// and returns default value if it's not.
func (k *Key) RangeTimeFormat(format string, defaultVal, min, max time.Time) time.Time {
val := k.MustTimeFormat(format)
if val.Unix() < min.Unix() || val.Unix() > max.Unix() {
return defaultVal
}
return val
}
// RangeTime checks if value with RFC3339 format is in given range inclusively,
// and returns default value if it's not.
func (k *Key) RangeTime(defaultVal, min, max time.Time) time.Time {
return k.RangeTimeFormat(time.RFC3339, defaultVal, min, max)
}
// Strings returns list of string divided by given delimiter.
func (k *Key) Strings(delim string) []string {
str := k.String()
if len(str) == 0 {
return []string{}
}
runes := []rune(str)
vals := make([]string, 0, 2)
var buf bytes.Buffer
escape := false
idx := 0
for {
if escape {
escape = false
if runes[idx] != '\\' && !strings.HasPrefix(string(runes[idx:]), delim) {
buf.WriteRune('\\')
}
buf.WriteRune(runes[idx])
} else {
if runes[idx] == '\\' {
escape = true
} else if strings.HasPrefix(string(runes[idx:]), delim) {
idx += len(delim) - 1
vals = append(vals, strings.TrimSpace(buf.String()))
buf.Reset()
} else {
buf.WriteRune(runes[idx])
}
}
idx += 1
if idx == len(runes) {
break
}
}
if buf.Len() > 0 {
vals = append(vals, strings.TrimSpace(buf.String()))
}
return vals
}
// StringsWithShadows returns list of string divided by given delimiter.
// Shadows will also be appended if any.
func (k *Key) StringsWithShadows(delim string) []string {
vals := k.ValueWithShadows()
results := make([]string, 0, len(vals)*2)
for i := range vals {
if len(vals) == 0 {
continue
}
results = append(results, strings.Split(vals[i], delim)...)
}
for i := range results {
results[i] = k.transformValue(strings.TrimSpace(results[i]))
}
return results
}
// Float64s returns list of float64 divided by given delimiter. Any invalid input will be treated as zero value.
func (k *Key) Float64s(delim string) []float64 {
vals, _ := k.parseFloat64s(k.Strings(delim), true, false)
return vals
}
// Ints returns list of int divided by given delimiter. Any invalid input will be treated as zero value.
func (k *Key) Ints(delim string) []int {
vals, _ := k.parseInts(k.Strings(delim), true, false)
return vals
}
// Int64s returns list of int64 divided by given delimiter. Any invalid input will be treated as zero value.
func (k *Key) Int64s(delim string) []int64 {
vals, _ := k.parseInt64s(k.Strings(delim), true, false)
return vals
}
// Uints returns list of uint divided by given delimiter. Any invalid input will be treated as zero value.
func (k *Key) Uints(delim string) []uint {
vals, _ := k.parseUints(k.Strings(delim), true, false)
return vals
}
// Uint64s returns list of uint64 divided by given delimiter. Any invalid input will be treated as zero value.
func (k *Key) Uint64s(delim string) []uint64 {
vals, _ := k.parseUint64s(k.Strings(delim), true, false)
return vals
}
// TimesFormat parses with given format and returns list of time.Time divided by given delimiter.
// Any invalid input will be treated as zero value (0001-01-01 00:00:00 +0000 UTC).
func (k *Key) TimesFormat(format, delim string) []time.Time {
vals, _ := k.parseTimesFormat(format, k.Strings(delim), true, false)
return vals
}
// Times parses with RFC3339 format and returns list of time.Time divided by given delimiter.
// Any invalid input will be treated as zero value (0001-01-01 00:00:00 +0000 UTC).
func (k *Key) Times(delim string) []time.Time {
return k.TimesFormat(time.RFC3339, delim)
}
// ValidFloat64s returns list of float64 divided by given delimiter. If some value is not float, then
// it will not be included to result list.
func (k *Key) ValidFloat64s(delim string) []float64 {
vals, _ := k.parseFloat64s(k.Strings(delim), false, false)
return vals
}
// ValidInts returns list of int divided by given delimiter. If some value is not integer, then it will
// not be included to result list.
func (k *Key) ValidInts(delim string) []int {
vals, _ := k.parseInts(k.Strings(delim), false, false)
return vals
}
// ValidInt64s returns list of int64 divided by given delimiter. If some value is not 64-bit integer,
// then it will not be included to result list.
func (k *Key) ValidInt64s(delim string) []int64 {
vals, _ := k.parseInt64s(k.Strings(delim), false, false)
return vals
}
// ValidUints returns list of uint divided by given delimiter. If some value is not unsigned integer,
// then it will not be included to result list.
func (k *Key) ValidUints(delim string) []uint {
vals, _ := k.parseUints(k.Strings(delim), false, false)
return vals
}
// ValidUint64s returns list of uint64 divided by given delimiter. If some value is not 64-bit unsigned
// integer, then it will not be included to result list.
func (k *Key) ValidUint64s(delim string) []uint64 {
vals, _ := k.parseUint64s(k.Strings(delim), false, false)
return vals
}
// ValidTimesFormat parses with given format and returns list of time.Time divided by given delimiter.
func (k *Key) ValidTimesFormat(format, delim string) []time.Time {
vals, _ := k.parseTimesFormat(format, k.Strings(delim), false, false)
return vals
}
// ValidTimes parses with RFC3339 format and returns list of time.Time divided by given delimiter.
func (k *Key) ValidTimes(delim string) []time.Time {
return k.ValidTimesFormat(time.RFC3339, delim)
}
// StrictFloat64s returns list of float64 divided by given delimiter or error on first invalid input.
func (k *Key) StrictFloat64s(delim string) ([]float64, error) {
return k.parseFloat64s(k.Strings(delim), false, true)
}
// StrictInts returns list of int divided by given delimiter or error on first invalid input.
func (k *Key) StrictInts(delim string) ([]int, error) {
return k.parseInts(k.Strings(delim), false, true)
}
// StrictInt64s returns list of int64 divided by given delimiter or error on first invalid input.
func (k *Key) StrictInt64s(delim string) ([]int64, error) {
return k.parseInt64s(k.Strings(delim), false, true)
}
// StrictUints returns list of uint divided by given delimiter or error on first invalid input.
func (k *Key) StrictUints(delim string) ([]uint, error) {
return k.parseUints(k.Strings(delim), false, true)
}
// StrictUint64s returns list of uint64 divided by given delimiter or error on first invalid input.
func (k *Key) StrictUint64s(delim string) ([]uint64, error) {
return k.parseUint64s(k.Strings(delim), false, true)
}
// StrictTimesFormat parses with given format and returns list of time.Time divided by given delimiter
// or error on first invalid input.
func (k *Key) StrictTimesFormat(format, delim string) ([]time.Time, error) {
return k.parseTimesFormat(format, k.Strings(delim), false, true)
}
// StrictTimes parses with RFC3339 format and returns list of time.Time divided by given delimiter
// or error on first invalid input.
func (k *Key) StrictTimes(delim string) ([]time.Time, error) {
return k.StrictTimesFormat(time.RFC3339, delim)
}
// parseFloat64s transforms strings to float64s.
func (k *Key) parseFloat64s(strs []string, addInvalid, returnOnInvalid bool) ([]float64, error) {
vals := make([]float64, 0, len(strs))
for _, str := range strs {
val, err := strconv.ParseFloat(str, 64)
if err != nil && returnOnInvalid {
return nil, err
}
if err == nil || addInvalid {
vals = append(vals, val)
}
}
return vals, nil
}
// parseInts transforms strings to ints.
func (k *Key) parseInts(strs []string, addInvalid, returnOnInvalid bool) ([]int, error) {
vals := make([]int, 0, len(strs))
for _, str := range strs {
val, err := strconv.Atoi(str)
if err != nil && returnOnInvalid {
return nil, err
}
if err == nil || addInvalid {
vals = append(vals, val)
}
}
return vals, nil
}
// parseInt64s transforms strings to int64s.
func (k *Key) parseInt64s(strs []string, addInvalid, returnOnInvalid bool) ([]int64, error) {
vals := make([]int64, 0, len(strs))
for _, str := range strs {
val, err := strconv.ParseInt(str, 10, 64)
if err != nil && returnOnInvalid {
return nil, err
}
if err == nil || addInvalid {
vals = append(vals, val)
}
}
return vals, nil
}
// parseUints transforms strings to uints.
func (k *Key) parseUints(strs []string, addInvalid, returnOnInvalid bool) ([]uint, error) {
vals := make([]uint, 0, len(strs))
for _, str := range strs {
val, err := strconv.ParseUint(str, 10, 0)
if err != nil && returnOnInvalid {
return nil, err
}
if err == nil || addInvalid {
vals = append(vals, uint(val))
}
}
return vals, nil
}
// parseUint64s transforms strings to uint64s.
func (k *Key) parseUint64s(strs []string, addInvalid, returnOnInvalid bool) ([]uint64, error) {
vals := make([]uint64, 0, len(strs))
for _, str := range strs {
val, err := strconv.ParseUint(str, 10, 64)
if err != nil && returnOnInvalid {
return nil, err
}
if err == nil || addInvalid {
vals = append(vals, val)
}
}
return vals, nil
}
// parseTimesFormat transforms strings to times in given format.
func (k *Key) parseTimesFormat(format string, strs []string, addInvalid, returnOnInvalid bool) ([]time.Time, error) {
vals := make([]time.Time, 0, len(strs))
for _, str := range strs {
val, err := time.Parse(format, str)
if err != nil && returnOnInvalid {
return nil, err
}
if err == nil || addInvalid {
vals = append(vals, val)
}
}
return vals, nil
}
// SetValue changes key value.
func (k *Key) SetValue(v string) {
if k.s.f.BlockMode {
k.s.f.lock.Lock()
defer k.s.f.lock.Unlock()
}
k.value = v
k.s.keysHash[k.name] = v
}

401
vendor/github.com/go-ini/ini/parser.go generated vendored Normal file
View file

@ -0,0 +1,401 @@
// Copyright 2015 Unknwon
//
// Licensed under the Apache License, Version 2.0 (the "License"): you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations
// under the License.
package ini
import (
"bufio"
"bytes"
"fmt"
"io"
"strconv"
"strings"
"unicode"
)
type tokenType int
const (
_TOKEN_INVALID tokenType = iota
_TOKEN_COMMENT
_TOKEN_SECTION
_TOKEN_KEY
)
type parser struct {
buf *bufio.Reader
isEOF bool
count int
comment *bytes.Buffer
}
func newParser(r io.Reader) *parser {
return &parser{
buf: bufio.NewReader(r),
count: 1,
comment: &bytes.Buffer{},
}
}
// BOM handles header of UTF-8, UTF-16 LE and UTF-16 BE's BOM format.
// http://en.wikipedia.org/wiki/Byte_order_mark#Representations_of_byte_order_marks_by_encoding
func (p *parser) BOM() error {
mask, err := p.buf.Peek(2)
if err != nil && err != io.EOF {
return err
} else if len(mask) < 2 {
return nil
}
switch {
case mask[0] == 254 && mask[1] == 255:
fallthrough
case mask[0] == 255 && mask[1] == 254:
p.buf.Read(mask)
case mask[0] == 239 && mask[1] == 187:
mask, err := p.buf.Peek(3)
if err != nil && err != io.EOF {
return err
} else if len(mask) < 3 {
return nil
}
if mask[2] == 191 {
p.buf.Read(mask)
}
}
return nil
}
func (p *parser) readUntil(delim byte) ([]byte, error) {
data, err := p.buf.ReadBytes(delim)
if err != nil {
if err == io.EOF {
p.isEOF = true
} else {
return nil, err
}
}
return data, nil
}
func cleanComment(in []byte) ([]byte, bool) {
i := bytes.IndexAny(in, "#;")
if i == -1 {
return nil, false
}
return in[i:], true
}
func readKeyName(in []byte) (string, int, error) {
line := string(in)
// Check if key name surrounded by quotes.
var keyQuote string
if line[0] == '"' {
if len(line) > 6 && string(line[0:3]) == `"""` {
keyQuote = `"""`
} else {
keyQuote = `"`
}
} else if line[0] == '`' {
keyQuote = "`"
}
// Get out key name
endIdx := -1
if len(keyQuote) > 0 {
startIdx := len(keyQuote)
// FIXME: fail case -> """"""name"""=value
pos := strings.Index(line[startIdx:], keyQuote)
if pos == -1 {
return "", -1, fmt.Errorf("missing closing key quote: %s", line)
}
pos += startIdx
// Find key-value delimiter
i := strings.IndexAny(line[pos+startIdx:], "=:")
if i < 0 {
return "", -1, ErrDelimiterNotFound{line}
}
endIdx = pos + i
return strings.TrimSpace(line[startIdx:pos]), endIdx + startIdx + 1, nil
}
endIdx = strings.IndexAny(line, "=:")
if endIdx < 0 {
return "", -1, ErrDelimiterNotFound{line}
}
return strings.TrimSpace(line[0:endIdx]), endIdx + 1, nil
}
func (p *parser) readMultilines(line, val, valQuote string) (string, error) {
for {
data, err := p.readUntil('\n')
if err != nil {
return "", err
}
next := string(data)
pos := strings.LastIndex(next, valQuote)
if pos > -1 {
val += next[:pos]
comment, has := cleanComment([]byte(next[pos:]))
if has {
p.comment.Write(bytes.TrimSpace(comment))
}
break
}
val += next
if p.isEOF {
return "", fmt.Errorf("missing closing key quote from '%s' to '%s'", line, next)
}
}
return val, nil
}
func (p *parser) readContinuationLines(val string) (string, error) {
for {
data, err := p.readUntil('\n')
if err != nil {
return "", err
}
next := strings.TrimSpace(string(data))
if len(next) == 0 {
break
}
val += next
if val[len(val)-1] != '\\' {
break
}
val = val[:len(val)-1]
}
return val, nil
}
// hasSurroundedQuote check if and only if the first and last characters
// are quotes \" or \'.
// It returns false if any other parts also contain same kind of quotes.
func hasSurroundedQuote(in string, quote byte) bool {
return len(in) >= 2 && in[0] == quote && in[len(in)-1] == quote &&
strings.IndexByte(in[1:], quote) == len(in)-2
}
func (p *parser) readValue(in []byte,
ignoreContinuation, ignoreInlineComment, unescapeValueDoubleQuotes, unescapeValueCommentSymbols bool) (string, error) {
line := strings.TrimLeftFunc(string(in), unicode.IsSpace)
if len(line) == 0 {
return "", nil
}
var valQuote string
if len(line) > 3 && string(line[0:3]) == `"""` {
valQuote = `"""`
} else if line[0] == '`' {
valQuote = "`"
} else if unescapeValueDoubleQuotes && line[0] == '"' {
valQuote = `"`
}
if len(valQuote) > 0 {
startIdx := len(valQuote)
pos := strings.LastIndex(line[startIdx:], valQuote)
// Check for multi-line value
if pos == -1 {
return p.readMultilines(line, line[startIdx:], valQuote)
}
if unescapeValueDoubleQuotes && valQuote == `"` {
return strings.Replace(line[startIdx:pos+startIdx], `\"`, `"`, -1), nil
}
return line[startIdx : pos+startIdx], nil
}
// Won't be able to reach here if value only contains whitespace
line = strings.TrimSpace(line)
// Check continuation lines when desired
if !ignoreContinuation && line[len(line)-1] == '\\' {
return p.readContinuationLines(line[:len(line)-1])
}
// Check if ignore inline comment
if !ignoreInlineComment {
i := strings.IndexAny(line, "#;")
if i > -1 {
p.comment.WriteString(line[i:])
line = strings.TrimSpace(line[:i])
}
}
// Trim single and double quotes
if hasSurroundedQuote(line, '\'') ||
hasSurroundedQuote(line, '"') {
line = line[1 : len(line)-1]
} else if len(valQuote) == 0 && unescapeValueCommentSymbols {
if strings.Contains(line, `\;`) {
line = strings.Replace(line, `\;`, ";", -1)
}
if strings.Contains(line, `\#`) {
line = strings.Replace(line, `\#`, "#", -1)
}
}
return line, nil
}
// parse parses data through an io.Reader.
func (f *File) parse(reader io.Reader) (err error) {
p := newParser(reader)
if err = p.BOM(); err != nil {
return fmt.Errorf("BOM: %v", err)
}
// Ignore error because default section name is never empty string.
name := DEFAULT_SECTION
if f.options.Insensitive {
name = strings.ToLower(DEFAULT_SECTION)
}
section, _ := f.NewSection(name)
// This "last" is not strictly equivalent to "previous one" if current key is not the first nested key
var isLastValueEmpty bool
var lastRegularKey *Key
var line []byte
var inUnparseableSection bool
for !p.isEOF {
line, err = p.readUntil('\n')
if err != nil {
return err
}
if f.options.AllowNestedValues &&
isLastValueEmpty && len(line) > 0 {
if line[0] == ' ' || line[0] == '\t' {
lastRegularKey.addNestedValue(string(bytes.TrimSpace(line)))
continue
}
}
line = bytes.TrimLeftFunc(line, unicode.IsSpace)
if len(line) == 0 {
continue
}
// Comments
if line[0] == '#' || line[0] == ';' {
// Note: we do not care ending line break,
// it is needed for adding second line,
// so just clean it once at the end when set to value.
p.comment.Write(line)
continue
}
// Section
if line[0] == '[' {
// Read to the next ']' (TODO: support quoted strings)
// TODO(unknwon): use LastIndexByte when stop supporting Go1.4
closeIdx := bytes.LastIndex(line, []byte("]"))
if closeIdx == -1 {
return fmt.Errorf("unclosed section: %s", line)
}
name := string(line[1:closeIdx])
section, err = f.NewSection(name)
if err != nil {
return err
}
comment, has := cleanComment(line[closeIdx+1:])
if has {
p.comment.Write(comment)
}
section.Comment = strings.TrimSpace(p.comment.String())
// Reset aotu-counter and comments
p.comment.Reset()
p.count = 1
inUnparseableSection = false
for i := range f.options.UnparseableSections {
if f.options.UnparseableSections[i] == name ||
(f.options.Insensitive && strings.ToLower(f.options.UnparseableSections[i]) == strings.ToLower(name)) {
inUnparseableSection = true
continue
}
}
continue
}
if inUnparseableSection {
section.isRawSection = true
section.rawBody += string(line)
continue
}
kname, offset, err := readKeyName(line)
if err != nil {
// Treat as boolean key when desired, and whole line is key name.
if IsErrDelimiterNotFound(err) && f.options.AllowBooleanKeys {
kname, err := p.readValue(line,
f.options.IgnoreContinuation,
f.options.IgnoreInlineComment,
f.options.UnescapeValueDoubleQuotes,
f.options.UnescapeValueCommentSymbols)
if err != nil {
return err
}
key, err := section.NewBooleanKey(kname)
if err != nil {
return err
}
key.Comment = strings.TrimSpace(p.comment.String())
p.comment.Reset()
continue
}
return err
}
// Auto increment.
isAutoIncr := false
if kname == "-" {
isAutoIncr = true
kname = "#" + strconv.Itoa(p.count)
p.count++
}
value, err := p.readValue(line[offset:],
f.options.IgnoreContinuation,
f.options.IgnoreInlineComment,
f.options.UnescapeValueDoubleQuotes,
f.options.UnescapeValueCommentSymbols)
if err != nil {
return err
}
isLastValueEmpty = len(value) == 0
key, err := section.NewKey(kname, value)
if err != nil {
return err
}
key.isAutoIncrement = isAutoIncr
key.Comment = strings.TrimSpace(p.comment.String())
p.comment.Reset()
lastRegularKey = key
}
return nil
}

257
vendor/github.com/go-ini/ini/section.go generated vendored Normal file
View file

@ -0,0 +1,257 @@
// Copyright 2014 Unknwon
//
// Licensed under the Apache License, Version 2.0 (the "License"): you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations
// under the License.
package ini
import (
"errors"
"fmt"
"strings"
)
// Section represents a config section.
type Section struct {
f *File
Comment string
name string
keys map[string]*Key
keyList []string
keysHash map[string]string
isRawSection bool
rawBody string
}
func newSection(f *File, name string) *Section {
return &Section{
f: f,
name: name,
keys: make(map[string]*Key),
keyList: make([]string, 0, 10),
keysHash: make(map[string]string),
}
}
// Name returns name of Section.
func (s *Section) Name() string {
return s.name
}
// Body returns rawBody of Section if the section was marked as unparseable.
// It still follows the other rules of the INI format surrounding leading/trailing whitespace.
func (s *Section) Body() string {
return strings.TrimSpace(s.rawBody)
}
// SetBody updates body content only if section is raw.
func (s *Section) SetBody(body string) {
if !s.isRawSection {
return
}
s.rawBody = body
}
// NewKey creates a new key to given section.
func (s *Section) NewKey(name, val string) (*Key, error) {
if len(name) == 0 {
return nil, errors.New("error creating new key: empty key name")
} else if s.f.options.Insensitive {
name = strings.ToLower(name)
}
if s.f.BlockMode {
s.f.lock.Lock()
defer s.f.lock.Unlock()
}
if inSlice(name, s.keyList) {
if s.f.options.AllowShadows {
if err := s.keys[name].addShadow(val); err != nil {
return nil, err
}
} else {
s.keys[name].value = val
}
return s.keys[name], nil
}
s.keyList = append(s.keyList, name)
s.keys[name] = newKey(s, name, val)
s.keysHash[name] = val
return s.keys[name], nil
}
// NewBooleanKey creates a new boolean type key to given section.
func (s *Section) NewBooleanKey(name string) (*Key, error) {
key, err := s.NewKey(name, "true")
if err != nil {
return nil, err
}
key.isBooleanType = true
return key, nil
}
// GetKey returns key in section by given name.
func (s *Section) GetKey(name string) (*Key, error) {
// FIXME: change to section level lock?
if s.f.BlockMode {
s.f.lock.RLock()
}
if s.f.options.Insensitive {
name = strings.ToLower(name)
}
key := s.keys[name]
if s.f.BlockMode {
s.f.lock.RUnlock()
}
if key == nil {
// Check if it is a child-section.
sname := s.name
for {
if i := strings.LastIndex(sname, "."); i > -1 {
sname = sname[:i]
sec, err := s.f.GetSection(sname)
if err != nil {
continue
}
return sec.GetKey(name)
} else {
break
}
}
return nil, fmt.Errorf("error when getting key of section '%s': key '%s' not exists", s.name, name)
}
return key, nil
}
// HasKey returns true if section contains a key with given name.
func (s *Section) HasKey(name string) bool {
key, _ := s.GetKey(name)
return key != nil
}
// Haskey is a backwards-compatible name for HasKey.
// TODO: delete me in v2
func (s *Section) Haskey(name string) bool {
return s.HasKey(name)
}
// HasValue returns true if section contains given raw value.
func (s *Section) HasValue(value string) bool {
if s.f.BlockMode {
s.f.lock.RLock()
defer s.f.lock.RUnlock()
}
for _, k := range s.keys {
if value == k.value {
return true
}
}
return false
}
// Key assumes named Key exists in section and returns a zero-value when not.
func (s *Section) Key(name string) *Key {
key, err := s.GetKey(name)
if err != nil {
// It's OK here because the only possible error is empty key name,
// but if it's empty, this piece of code won't be executed.
key, _ = s.NewKey(name, "")
return key
}
return key
}
// Keys returns list of keys of section.
func (s *Section) Keys() []*Key {
keys := make([]*Key, len(s.keyList))
for i := range s.keyList {
keys[i] = s.Key(s.keyList[i])
}
return keys
}
// ParentKeys returns list of keys of parent section.
func (s *Section) ParentKeys() []*Key {
var parentKeys []*Key
sname := s.name
for {
if i := strings.LastIndex(sname, "."); i > -1 {
sname = sname[:i]
sec, err := s.f.GetSection(sname)
if err != nil {
continue
}
parentKeys = append(parentKeys, sec.Keys()...)
} else {
break
}
}
return parentKeys
}
// KeyStrings returns list of key names of section.
func (s *Section) KeyStrings() []string {
list := make([]string, len(s.keyList))
copy(list, s.keyList)
return list
}
// KeysHash returns keys hash consisting of names and values.
func (s *Section) KeysHash() map[string]string {
if s.f.BlockMode {
s.f.lock.RLock()
defer s.f.lock.RUnlock()
}
hash := map[string]string{}
for key, value := range s.keysHash {
hash[key] = value
}
return hash
}
// DeleteKey deletes a key from section.
func (s *Section) DeleteKey(name string) {
if s.f.BlockMode {
s.f.lock.Lock()
defer s.f.lock.Unlock()
}
for i, k := range s.keyList {
if k == name {
s.keyList = append(s.keyList[:i], s.keyList[i+1:]...)
delete(s.keys, name)
return
}
}
}
// ChildSections returns a list of child sections of current section.
// For example, "[parent.child1]" and "[parent.child12]" are child sections
// of section "[parent]".
func (s *Section) ChildSections() []*Section {
prefix := s.name + "."
children := make([]*Section, 0, 3)
for _, name := range s.f.sectionList {
if strings.HasPrefix(name, prefix) {
children = append(children, s.f.sections[name])
}
}
return children
}

View file

@ -19,6 +19,7 @@ import (
"errors"
"fmt"
"reflect"
"strings"
"time"
"unicode"
)
@ -76,10 +77,80 @@ func parseDelim(actual string) string {
var reflectTime = reflect.TypeOf(time.Now()).Kind()
// setSliceWithProperType sets proper values to slice based on its type.
func setSliceWithProperType(key *Key, field reflect.Value, delim string, allowShadow, isStrict bool) error {
var strs []string
if allowShadow {
strs = key.StringsWithShadows(delim)
} else {
strs = key.Strings(delim)
}
numVals := len(strs)
if numVals == 0 {
return nil
}
var vals interface{}
var err error
sliceOf := field.Type().Elem().Kind()
switch sliceOf {
case reflect.String:
vals = strs
case reflect.Int:
vals, err = key.parseInts(strs, true, false)
case reflect.Int64:
vals, err = key.parseInt64s(strs, true, false)
case reflect.Uint:
vals, err = key.parseUints(strs, true, false)
case reflect.Uint64:
vals, err = key.parseUint64s(strs, true, false)
case reflect.Float64:
vals, err = key.parseFloat64s(strs, true, false)
case reflectTime:
vals, err = key.parseTimesFormat(time.RFC3339, strs, true, false)
default:
return fmt.Errorf("unsupported type '[]%s'", sliceOf)
}
if err != nil && isStrict {
return err
}
slice := reflect.MakeSlice(field.Type(), numVals, numVals)
for i := 0; i < numVals; i++ {
switch sliceOf {
case reflect.String:
slice.Index(i).Set(reflect.ValueOf(vals.([]string)[i]))
case reflect.Int:
slice.Index(i).Set(reflect.ValueOf(vals.([]int)[i]))
case reflect.Int64:
slice.Index(i).Set(reflect.ValueOf(vals.([]int64)[i]))
case reflect.Uint:
slice.Index(i).Set(reflect.ValueOf(vals.([]uint)[i]))
case reflect.Uint64:
slice.Index(i).Set(reflect.ValueOf(vals.([]uint64)[i]))
case reflect.Float64:
slice.Index(i).Set(reflect.ValueOf(vals.([]float64)[i]))
case reflectTime:
slice.Index(i).Set(reflect.ValueOf(vals.([]time.Time)[i]))
}
}
field.Set(slice)
return nil
}
func wrapStrictError(err error, isStrict bool) error {
if isStrict {
return err
}
return nil
}
// setWithProperType sets proper value to field based on its type,
// but it does not return error for failing parsing,
// because we want to use default value that is already assigned to strcut.
func setWithProperType(t reflect.Type, key *Key, field reflect.Value, delim string) error {
func setWithProperType(t reflect.Type, key *Key, field reflect.Value, delim string, allowShadow, isStrict bool) error {
switch t.Kind() {
case reflect.String:
if len(key.String()) == 0 {
@ -89,78 +160,70 @@ func setWithProperType(t reflect.Type, key *Key, field reflect.Value, delim stri
case reflect.Bool:
boolVal, err := key.Bool()
if err != nil {
return nil
return wrapStrictError(err, isStrict)
}
field.SetBool(boolVal)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
durationVal, err := key.Duration()
if err == nil {
// Skip zero value
if err == nil && int64(durationVal) > 0 {
field.Set(reflect.ValueOf(durationVal))
return nil
}
intVal, err := key.Int64()
if err != nil {
return nil
return wrapStrictError(err, isStrict)
}
field.SetInt(intVal)
// byte is an alias for uint8, so supporting uint8 breaks support for byte
case reflect.Uint, reflect.Uint16, reflect.Uint32, reflect.Uint64:
durationVal, err := key.Duration()
if err == nil {
// Skip zero value
if err == nil && int(durationVal) > 0 {
field.Set(reflect.ValueOf(durationVal))
return nil
}
uintVal, err := key.Uint64()
if err != nil {
return nil
return wrapStrictError(err, isStrict)
}
field.SetUint(uintVal)
case reflect.Float64:
case reflect.Float32, reflect.Float64:
floatVal, err := key.Float64()
if err != nil {
return nil
return wrapStrictError(err, isStrict)
}
field.SetFloat(floatVal)
case reflectTime:
timeVal, err := key.Time()
if err != nil {
return nil
return wrapStrictError(err, isStrict)
}
field.Set(reflect.ValueOf(timeVal))
case reflect.Slice:
vals := key.Strings(delim)
numVals := len(vals)
if numVals == 0 {
return nil
}
sliceOf := field.Type().Elem().Kind()
var times []time.Time
if sliceOf == reflectTime {
times = key.Times(delim)
}
slice := reflect.MakeSlice(field.Type(), numVals, numVals)
for i := 0; i < numVals; i++ {
switch sliceOf {
case reflectTime:
slice.Index(i).Set(reflect.ValueOf(times[i]))
default:
slice.Index(i).Set(reflect.ValueOf(vals[i]))
}
}
field.Set(slice)
return setSliceWithProperType(key, field, delim, allowShadow, isStrict)
default:
return fmt.Errorf("unsupported type '%s'", t)
}
return nil
}
func (s *Section) mapTo(val reflect.Value) error {
func parseTagOptions(tag string) (rawName string, omitEmpty bool, allowShadow bool) {
opts := strings.SplitN(tag, ",", 3)
rawName = opts[0]
if len(opts) > 1 {
omitEmpty = opts[1] == "omitempty"
}
if len(opts) > 2 {
allowShadow = opts[2] == "allowshadow"
}
return rawName, omitEmpty, allowShadow
}
func (s *Section) mapTo(val reflect.Value, isStrict bool) error {
if val.Kind() == reflect.Ptr {
val = val.Elem()
}
@ -175,7 +238,8 @@ func (s *Section) mapTo(val reflect.Value) error {
continue
}
fieldName := s.parseFieldName(tpField.Name, tag)
rawName, _, allowShadow := parseTagOptions(tag)
fieldName := s.parseFieldName(tpField.Name, rawName)
if len(fieldName) == 0 || !field.CanSet() {
continue
}
@ -188,7 +252,7 @@ func (s *Section) mapTo(val reflect.Value) error {
if isAnonymous || isStruct {
if sec, err := s.f.GetSection(fieldName); err == nil {
if err = sec.mapTo(field); err != nil {
if err = sec.mapTo(field, isStrict); err != nil {
return fmt.Errorf("error mapping field(%s): %v", fieldName, err)
}
continue
@ -196,7 +260,8 @@ func (s *Section) mapTo(val reflect.Value) error {
}
if key, err := s.GetKey(fieldName); err == nil {
if err = setWithProperType(tpField.Type, key, field, parseDelim(tpField.Tag.Get("delim"))); err != nil {
delim := parseDelim(tpField.Tag.Get("delim"))
if err = setWithProperType(tpField.Type, key, field, delim, allowShadow, isStrict); err != nil {
return fmt.Errorf("error mapping field(%s): %v", fieldName, err)
}
}
@ -215,7 +280,22 @@ func (s *Section) MapTo(v interface{}) error {
return errors.New("cannot map to non-pointer struct")
}
return s.mapTo(val)
return s.mapTo(val, false)
}
// MapTo maps section to given struct in strict mode,
// which returns all possible error including value parsing error.
func (s *Section) StrictMapTo(v interface{}) error {
typ := reflect.TypeOf(v)
val := reflect.ValueOf(v)
if typ.Kind() == reflect.Ptr {
typ = typ.Elem()
val = val.Elem()
} else {
return errors.New("cannot map to non-pointer struct")
}
return s.mapTo(val, true)
}
// MapTo maps file to given struct.
@ -223,6 +303,12 @@ func (f *File) MapTo(v interface{}) error {
return f.Section("").MapTo(v)
}
// MapTo maps file to given struct in strict mode,
// which returns all possible error including value parsing error.
func (f *File) StrictMapTo(v interface{}) error {
return f.Section("").StrictMapTo(v)
}
// MapTo maps data sources to given struct with name mapper.
func MapToWithMapper(v interface{}, mapper NameMapper, source interface{}, others ...interface{}) error {
cfg, err := Load(source, others...)
@ -233,45 +319,104 @@ func MapToWithMapper(v interface{}, mapper NameMapper, source interface{}, other
return cfg.MapTo(v)
}
// StrictMapToWithMapper maps data sources to given struct with name mapper in strict mode,
// which returns all possible error including value parsing error.
func StrictMapToWithMapper(v interface{}, mapper NameMapper, source interface{}, others ...interface{}) error {
cfg, err := Load(source, others...)
if err != nil {
return err
}
cfg.NameMapper = mapper
return cfg.StrictMapTo(v)
}
// MapTo maps data sources to given struct.
func MapTo(v, source interface{}, others ...interface{}) error {
return MapToWithMapper(v, nil, source, others...)
}
// reflectWithProperType does the opposite thing with setWithProperType.
// StrictMapTo maps data sources to given struct in strict mode,
// which returns all possible error including value parsing error.
func StrictMapTo(v, source interface{}, others ...interface{}) error {
return StrictMapToWithMapper(v, nil, source, others...)
}
// reflectSliceWithProperType does the opposite thing as setSliceWithProperType.
func reflectSliceWithProperType(key *Key, field reflect.Value, delim string) error {
slice := field.Slice(0, field.Len())
if field.Len() == 0 {
return nil
}
var buf bytes.Buffer
sliceOf := field.Type().Elem().Kind()
for i := 0; i < field.Len(); i++ {
switch sliceOf {
case reflect.String:
buf.WriteString(slice.Index(i).String())
case reflect.Int, reflect.Int64:
buf.WriteString(fmt.Sprint(slice.Index(i).Int()))
case reflect.Uint, reflect.Uint64:
buf.WriteString(fmt.Sprint(slice.Index(i).Uint()))
case reflect.Float64:
buf.WriteString(fmt.Sprint(slice.Index(i).Float()))
case reflectTime:
buf.WriteString(slice.Index(i).Interface().(time.Time).Format(time.RFC3339))
default:
return fmt.Errorf("unsupported type '[]%s'", sliceOf)
}
buf.WriteString(delim)
}
key.SetValue(buf.String()[:buf.Len()-1])
return nil
}
// reflectWithProperType does the opposite thing as setWithProperType.
func reflectWithProperType(t reflect.Type, key *Key, field reflect.Value, delim string) error {
switch t.Kind() {
case reflect.String:
key.SetValue(field.String())
case reflect.Bool,
reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64,
reflect.Float64,
reflectTime:
key.SetValue(fmt.Sprint(field))
case reflect.Bool:
key.SetValue(fmt.Sprint(field.Bool()))
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
key.SetValue(fmt.Sprint(field.Int()))
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
key.SetValue(fmt.Sprint(field.Uint()))
case reflect.Float32, reflect.Float64:
key.SetValue(fmt.Sprint(field.Float()))
case reflectTime:
key.SetValue(fmt.Sprint(field.Interface().(time.Time).Format(time.RFC3339)))
case reflect.Slice:
vals := field.Slice(0, field.Len())
if field.Len() == 0 {
return nil
}
var buf bytes.Buffer
isTime := fmt.Sprint(field.Type()) == "[]time.Time"
for i := 0; i < field.Len(); i++ {
if isTime {
buf.WriteString(vals.Index(i).Interface().(time.Time).Format(time.RFC3339))
} else {
buf.WriteString(fmt.Sprint(vals.Index(i)))
}
buf.WriteString(delim)
}
key.SetValue(buf.String()[:buf.Len()-1])
return reflectSliceWithProperType(key, field, delim)
default:
return fmt.Errorf("unsupported type '%s'", t)
}
return nil
}
// CR: copied from encoding/json/encode.go with modifications of time.Time support.
// TODO: add more test coverage.
func isEmptyValue(v reflect.Value) bool {
switch v.Kind() {
case reflect.Array, reflect.Map, reflect.Slice, reflect.String:
return v.Len() == 0
case reflect.Bool:
return !v.Bool()
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return v.Int() == 0
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return v.Uint() == 0
case reflect.Float32, reflect.Float64:
return v.Float() == 0
case reflect.Interface, reflect.Ptr:
return v.IsNil()
case reflectTime:
t, ok := v.Interface().(time.Time)
return ok && t.IsZero()
}
return false
}
func (s *Section) reflectFrom(val reflect.Value) error {
if val.Kind() == reflect.Ptr {
val = val.Elem()
@ -287,21 +432,32 @@ func (s *Section) reflectFrom(val reflect.Value) error {
continue
}
fieldName := s.parseFieldName(tpField.Name, tag)
opts := strings.SplitN(tag, ",", 2)
if len(opts) == 2 && opts[1] == "omitempty" && isEmptyValue(field) {
continue
}
fieldName := s.parseFieldName(tpField.Name, opts[0])
if len(fieldName) == 0 || !field.CanSet() {
continue
}
if (tpField.Type.Kind() == reflect.Ptr && tpField.Anonymous) ||
(tpField.Type.Kind() == reflect.Struct) {
(tpField.Type.Kind() == reflect.Struct && tpField.Type.Name() != "Time") {
// Note: The only error here is section doesn't exist.
sec, err := s.f.GetSection(fieldName)
if err != nil {
// Note: fieldName can never be empty here, ignore error.
sec, _ = s.f.NewSection(fieldName)
}
// Add comment from comment tag
if len(sec.Comment) == 0 {
sec.Comment = tpField.Tag.Get("comment")
}
if err = sec.reflectFrom(field); err != nil {
return fmt.Errorf("error reflecting field(%s): %v", fieldName, err)
return fmt.Errorf("error reflecting field (%s): %v", fieldName, err)
}
continue
}
@ -311,8 +467,14 @@ func (s *Section) reflectFrom(val reflect.Value) error {
if err != nil {
key, _ = s.NewKey(fieldName, "")
}
// Add comment from comment tag
if len(key.Comment) == 0 {
key.Comment = tpField.Tag.Get("comment")
}
if err = reflectWithProperType(tpField.Type, key, field, parseDelim(tpField.Tag.Get("delim"))); err != nil {
return fmt.Errorf("error reflecting field(%s): %v", fieldName, err)
return fmt.Errorf("error reflecting field (%s): %v", fieldName, err)
}
}

View file

@ -1,9 +0,0 @@
language: go
go:
- 1.4.3
- 1.5.3
- tip
script:
- go test -v ./...

View file

@ -42,7 +42,7 @@ func NewDCESecurity(domain Domain, id uint32) (UUID, error) {
// NewDCEPerson returns a DCE Security (Version 2) UUID in the person
// domain with the id returned by os.Getuid.
//
// NewDCEPerson(Person, uint32(os.Getuid()))
// NewDCESecurity(Person, uint32(os.Getuid()))
func NewDCEPerson() (UUID, error) {
return NewDCESecurity(Person, uint32(os.Getuid()))
}
@ -50,7 +50,7 @@ func NewDCEPerson() (UUID, error) {
// NewDCEGroup returns a DCE Security (Version 2) UUID in the group
// domain with the id returned by os.Getgid.
//
// NewDCEGroup(Group, uint32(os.Getgid()))
// NewDCESecurity(Group, uint32(os.Getgid()))
func NewDCEGroup() (UUID, error) {
return NewDCESecurity(Group, uint32(os.Getgid()))
}

View file

@ -27,7 +27,7 @@ var (
func NewHash(h hash.Hash, space UUID, data []byte, version int) UUID {
h.Reset()
h.Write(space[:])
h.Write([]byte(data))
h.Write(data)
s := h.Sum(nil)
var uuid UUID
copy(uuid[:], s)

View file

@ -1,62 +0,0 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"encoding/json"
"reflect"
"testing"
)
var testUUID = Must(Parse("f47ac10b-58cc-0372-8567-0e02b2c3d479"))
func TestJSON(t *testing.T) {
type S struct {
ID1 UUID
ID2 UUID
}
s1 := S{ID1: testUUID}
data, err := json.Marshal(&s1)
if err != nil {
t.Fatal(err)
}
var s2 S
if err := json.Unmarshal(data, &s2); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(&s1, &s2) {
t.Errorf("got %#v, want %#v", s2, s1)
}
}
func BenchmarkUUID_MarshalJSON(b *testing.B) {
x := &struct {
UUID UUID `json:"uuid"`
}{}
var err error
x.UUID, err = Parse("f47ac10b-58cc-0372-8567-0e02b2c3d479")
if err != nil {
b.Fatal(err)
}
for i := 0; i < b.N; i++ {
js, err := json.Marshal(x)
if err != nil {
b.Fatalf("marshal json: %#v (%v)", js, err)
}
}
}
func BenchmarkUUID_UnmarshalJSON(b *testing.B) {
js := []byte(`{"uuid":"f47ac10b-58cc-0372-8567-0e02b2c3d479"}`)
var x *struct {
UUID UUID `json:"uuid"`
}
for i := 0; i < b.N; i++ {
err := json.Unmarshal(js, &x)
if err != nil {
b.Fatalf("marshal json: %#v (%v)", js, err)
}
}
}

View file

@ -5,16 +5,14 @@
package uuid
import (
"net"
"sync"
)
var (
nodeMu sync.Mutex
interfaces []net.Interface // cached list of interfaces
ifname string // name of interface being used
nodeID [6]byte // hardware for version 1 UUIDs
zeroID [6]byte // nodeID with only 0's
nodeMu sync.Mutex
ifname string // name of interface being used
nodeID [6]byte // hardware for version 1 UUIDs
zeroID [6]byte // nodeID with only 0's
)
// NodeInterface returns the name of the interface from which the NodeID was
@ -39,20 +37,12 @@ func SetNodeInterface(name string) bool {
}
func setNodeInterface(name string) bool {
if interfaces == nil {
var err error
interfaces, err = net.Interfaces()
if err != nil && name != "" {
return false
}
}
for _, ifs := range interfaces {
if len(ifs.HardwareAddr) >= 6 && (name == "" || name == ifs.Name) {
copy(nodeID[:], ifs.HardwareAddr)
ifname = ifs.Name
return true
}
iname, addr := getHardwareInterface(name) // null implementation for js
if iname != "" && addr != nil {
ifname = iname
copy(nodeID[:], addr)
return true
}
// We found no interfaces with a valid hardware address. If name
@ -94,9 +84,6 @@ func SetNodeID(id []byte) bool {
// NodeID returns the 6 byte node id encoded in uuid. It returns nil if uuid is
// not valid. The NodeID is only well defined for version 1 and 2 UUIDs.
func (uuid UUID) NodeID() []byte {
if len(uuid) != 16 {
return nil
}
var node [6]byte
copy(node[:], uuid[10:])
return node[:]

12
vendor/github.com/google/uuid/node_js.go generated vendored Normal file
View file

@ -0,0 +1,12 @@
// Copyright 2017 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build js
package uuid
// getHardwareInterface returns nil values for the JS version of the code.
// This remvoves the "net" dependency, because it is not used in the browser.
// Using the "net" library inflates the size of the transpiled JS code by 673k bytes.
func getHardwareInterface(name string) (string, []byte) { return "", nil }

33
vendor/github.com/google/uuid/node_net.go generated vendored Normal file
View file

@ -0,0 +1,33 @@
// Copyright 2017 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !js
package uuid
import "net"
var interfaces []net.Interface // cached list of interfaces
// getHardwareInterface returns the name and hardware address of interface name.
// If name is "" then the name and hardware address of one of the system's
// interfaces is returned. If no interfaces are found (name does not exist or
// there are no interfaces) then "", nil is returned.
//
// Only addresses of at least 6 bytes are returned.
func getHardwareInterface(name string) (string, []byte) {
if interfaces == nil {
var err error
interfaces, err = net.Interfaces()
if err != nil {
return "", nil
}
}
for _, ifs := range interfaces {
if len(ifs.HardwareAddr) >= 6 && (name == "" || name == ifs.Name) {
return ifs.Name, ifs.HardwareAddr
}
}
return "", nil
}

View file

@ -1,66 +0,0 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"flag"
"runtime"
"testing"
"time"
)
// This test is only run when --regressions is passed on the go test line.
var regressions = flag.Bool("regressions", false, "run uuid regression tests")
// TestClockSeqRace tests for a particular race condition of returning two
// identical Version1 UUIDs. The duration of 1 minute was chosen as the race
// condition, before being fixed, nearly always occured in under 30 seconds.
func TestClockSeqRace(t *testing.T) {
if !*regressions {
t.Skip("skipping regression tests")
}
duration := time.Minute
done := make(chan struct{})
defer close(done)
ch := make(chan UUID, 10000)
ncpu := runtime.NumCPU()
switch ncpu {
case 0, 1:
// We can't run the test effectively.
t.Skip("skipping race test, only one CPU detected")
return
default:
runtime.GOMAXPROCS(ncpu)
}
for i := 0; i < ncpu; i++ {
go func() {
for {
select {
case <-done:
return
case ch <- Must(NewUUID()):
}
}
}()
}
uuids := make(map[string]bool)
cnt := 0
start := time.Now()
for u := range ch {
s := u.String()
if uuids[s] {
t.Errorf("duplicate uuid after %d in %v: %s", cnt, time.Since(start), s)
return
}
uuids[s] = true
if time.Since(start) > duration {
return
}
cnt++
}
}

View file

@ -1,113 +0,0 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"strings"
"testing"
)
func TestScan(t *testing.T) {
var stringTest string = "f47ac10b-58cc-0372-8567-0e02b2c3d479"
var badTypeTest int = 6
var invalidTest string = "f47ac10b-58cc-0372-8567-0e02b2c3d4"
byteTest := make([]byte, 16)
byteTestUUID := Must(Parse(stringTest))
copy(byteTest, byteTestUUID[:])
// sunny day tests
var uuid UUID
err := (&uuid).Scan(stringTest)
if err != nil {
t.Fatal(err)
}
err = (&uuid).Scan([]byte(stringTest))
if err != nil {
t.Fatal(err)
}
err = (&uuid).Scan(byteTest)
if err != nil {
t.Fatal(err)
}
// bad type tests
err = (&uuid).Scan(badTypeTest)
if err == nil {
t.Error("int correctly parsed and shouldn't have")
}
if !strings.Contains(err.Error(), "unable to scan type") {
t.Error("attempting to parse an int returned an incorrect error message")
}
// invalid/incomplete uuids
err = (&uuid).Scan(invalidTest)
if err == nil {
t.Error("invalid uuid was parsed without error")
}
if !strings.Contains(err.Error(), "invalid UUID") {
t.Error("attempting to parse an invalid UUID returned an incorrect error message")
}
err = (&uuid).Scan(byteTest[:len(byteTest)-2])
if err == nil {
t.Error("invalid byte uuid was parsed without error")
}
if !strings.Contains(err.Error(), "invalid UUID") {
t.Error("attempting to parse an invalid byte UUID returned an incorrect error message")
}
// empty tests
uuid = UUID{}
var emptySlice []byte
err = (&uuid).Scan(emptySlice)
if err != nil {
t.Fatal(err)
}
for _, v := range uuid {
if v != 0 {
t.Error("UUID was not nil after scanning empty byte slice")
}
}
uuid = UUID{}
var emptyString string
err = (&uuid).Scan(emptyString)
if err != nil {
t.Fatal(err)
}
for _, v := range uuid {
if v != 0 {
t.Error("UUID was not nil after scanning empty byte slice")
}
}
uuid = UUID{}
err = (&uuid).Scan(nil)
if err != nil {
t.Fatal(err)
}
for _, v := range uuid {
if v != 0 {
t.Error("UUID was not nil after scanning nil")
}
}
}
func TestValue(t *testing.T) {
stringTest := "f47ac10b-58cc-0372-8567-0e02b2c3d479"
uuid := Must(Parse(stringTest))
val, _ := uuid.Value()
if val != stringTest {
t.Error("Value() did not return expected string")
}
}

View file

@ -86,7 +86,7 @@ func clockSequence() int {
return int(clockSeq & 0x3fff)
}
// SetClockSeq sets the clock sequence to the lower 14 bits of seq. Setting to
// SetClockSequence sets the clock sequence to the lower 14 bits of seq. Setting to
// -1 causes a new sequence to be generated.
func SetClockSequence(seq int) {
defer timeMu.Unlock()
@ -100,9 +100,9 @@ func setClockSequence(seq int) {
randomBits(b[:]) // clock sequence
seq = int(b[0])<<8 | int(b[1])
}
old_seq := clockSeq
oldSeq := clockSeq
clockSeq = uint16(seq&0x3fff) | 0x8000 // Set our variant
if old_seq != clockSeq {
if oldSeq != clockSeq {
lasttime = 0
}
}

View file

@ -58,11 +58,11 @@ func Parse(s string) (UUID, error) {
14, 16,
19, 21,
24, 26, 28, 30, 32, 34} {
if v, ok := xtob(s[x], s[x+1]); !ok {
v, ok := xtob(s[x], s[x+1])
if !ok {
return uuid, errors.New("invalid UUID format")
} else {
uuid[i] = v
}
uuid[i] = v
}
return uuid, nil
}
@ -88,15 +88,22 @@ func ParseBytes(b []byte) (UUID, error) {
14, 16,
19, 21,
24, 26, 28, 30, 32, 34} {
if v, ok := xtob(b[x], b[x+1]); !ok {
v, ok := xtob(b[x], b[x+1])
if !ok {
return uuid, errors.New("invalid UUID format")
} else {
uuid[i] = v
}
uuid[i] = v
}
return uuid, nil
}
// FromBytes creates a new UUID from a byte slice. Returns an error if the slice
// does not have a length of 16. The bytes are copied from the slice.
func FromBytes(b []byte) (uuid UUID, err error) {
err = uuid.UnmarshalBinary(b)
return uuid, err
}
// Must returns uuid if err is nil and panics otherwise.
func Must(uuid UUID, err error) UUID {
if err != nil {
@ -176,7 +183,7 @@ func (v Variant) String() string {
return fmt.Sprintf("BadVariant%d", int(v))
}
// SetRand sets the random number generator to r, which implents io.Reader.
// SetRand sets the random number generator to r, which implements io.Reader.
// If r.Read returns an error when the package requests random data then
// a panic will be issued.
//

View file

@ -1,525 +0,0 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"bytes"
"fmt"
"os"
"strings"
"testing"
"time"
"unsafe"
)
type test struct {
in string
version Version
variant Variant
isuuid bool
}
var tests = []test{
{"f47ac10b-58cc-0372-8567-0e02b2c3d479", 0, RFC4122, true},
{"f47ac10b-58cc-1372-8567-0e02b2c3d479", 1, RFC4122, true},
{"f47ac10b-58cc-2372-8567-0e02b2c3d479", 2, RFC4122, true},
{"f47ac10b-58cc-3372-8567-0e02b2c3d479", 3, RFC4122, true},
{"f47ac10b-58cc-4372-8567-0e02b2c3d479", 4, RFC4122, true},
{"f47ac10b-58cc-5372-8567-0e02b2c3d479", 5, RFC4122, true},
{"f47ac10b-58cc-6372-8567-0e02b2c3d479", 6, RFC4122, true},
{"f47ac10b-58cc-7372-8567-0e02b2c3d479", 7, RFC4122, true},
{"f47ac10b-58cc-8372-8567-0e02b2c3d479", 8, RFC4122, true},
{"f47ac10b-58cc-9372-8567-0e02b2c3d479", 9, RFC4122, true},
{"f47ac10b-58cc-a372-8567-0e02b2c3d479", 10, RFC4122, true},
{"f47ac10b-58cc-b372-8567-0e02b2c3d479", 11, RFC4122, true},
{"f47ac10b-58cc-c372-8567-0e02b2c3d479", 12, RFC4122, true},
{"f47ac10b-58cc-d372-8567-0e02b2c3d479", 13, RFC4122, true},
{"f47ac10b-58cc-e372-8567-0e02b2c3d479", 14, RFC4122, true},
{"f47ac10b-58cc-f372-8567-0e02b2c3d479", 15, RFC4122, true},
{"urn:uuid:f47ac10b-58cc-4372-0567-0e02b2c3d479", 4, Reserved, true},
{"URN:UUID:f47ac10b-58cc-4372-0567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-0567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-1567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-2567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-3567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-4567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-5567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-6567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-7567-0e02b2c3d479", 4, Reserved, true},
{"f47ac10b-58cc-4372-8567-0e02b2c3d479", 4, RFC4122, true},
{"f47ac10b-58cc-4372-9567-0e02b2c3d479", 4, RFC4122, true},
{"f47ac10b-58cc-4372-a567-0e02b2c3d479", 4, RFC4122, true},
{"f47ac10b-58cc-4372-b567-0e02b2c3d479", 4, RFC4122, true},
{"f47ac10b-58cc-4372-c567-0e02b2c3d479", 4, Microsoft, true},
{"f47ac10b-58cc-4372-d567-0e02b2c3d479", 4, Microsoft, true},
{"f47ac10b-58cc-4372-e567-0e02b2c3d479", 4, Future, true},
{"f47ac10b-58cc-4372-f567-0e02b2c3d479", 4, Future, true},
{"f47ac10b158cc-5372-a567-0e02b2c3d479", 0, Invalid, false},
{"f47ac10b-58cc25372-a567-0e02b2c3d479", 0, Invalid, false},
{"f47ac10b-58cc-53723a567-0e02b2c3d479", 0, Invalid, false},
{"f47ac10b-58cc-5372-a56740e02b2c3d479", 0, Invalid, false},
{"f47ac10b-58cc-5372-a567-0e02-2c3d479", 0, Invalid, false},
{"g47ac10b-58cc-4372-a567-0e02b2c3d479", 0, Invalid, false},
}
var constants = []struct {
c interface{}
name string
}{
{Person, "Person"},
{Group, "Group"},
{Org, "Org"},
{Invalid, "Invalid"},
{RFC4122, "RFC4122"},
{Reserved, "Reserved"},
{Microsoft, "Microsoft"},
{Future, "Future"},
{Domain(17), "Domain17"},
{Variant(42), "BadVariant42"},
}
func testTest(t *testing.T, in string, tt test) {
uuid, err := Parse(in)
if ok := (err == nil); ok != tt.isuuid {
t.Errorf("Parse(%s) got %v expected %v\b", in, ok, tt.isuuid)
}
if err != nil {
return
}
if v := uuid.Variant(); v != tt.variant {
t.Errorf("Variant(%s) got %d expected %d\b", in, v, tt.variant)
}
if v := uuid.Version(); v != tt.version {
t.Errorf("Version(%s) got %d expected %d\b", in, v, tt.version)
}
}
func testBytes(t *testing.T, in []byte, tt test) {
uuid, err := ParseBytes(in)
if ok := (err == nil); ok != tt.isuuid {
t.Errorf("ParseBytes(%s) got %v expected %v\b", in, ok, tt.isuuid)
}
if err != nil {
return
}
suuid, _ := Parse(string(in))
if uuid != suuid {
t.Errorf("ParseBytes(%s) got %v expected %v\b", in, uuid, suuid)
}
}
func TestUUID(t *testing.T) {
for _, tt := range tests {
testTest(t, tt.in, tt)
testTest(t, strings.ToUpper(tt.in), tt)
testBytes(t, []byte(tt.in), tt)
}
}
func TestConstants(t *testing.T) {
for x, tt := range constants {
v, ok := tt.c.(fmt.Stringer)
if !ok {
t.Errorf("%x: %v: not a stringer", x, v)
} else if s := v.String(); s != tt.name {
v, _ := tt.c.(int)
t.Errorf("%x: Constant %T:%d gives %q, expected %q", x, tt.c, v, s, tt.name)
}
}
}
func TestRandomUUID(t *testing.T) {
m := make(map[string]bool)
for x := 1; x < 32; x++ {
uuid := New()
s := uuid.String()
if m[s] {
t.Errorf("NewRandom returned duplicated UUID %s", s)
}
m[s] = true
if v := uuid.Version(); v != 4 {
t.Errorf("Random UUID of version %s", v)
}
if uuid.Variant() != RFC4122 {
t.Errorf("Random UUID is variant %d", uuid.Variant())
}
}
}
func TestNew(t *testing.T) {
m := make(map[UUID]bool)
for x := 1; x < 32; x++ {
s := New()
if m[s] {
t.Errorf("New returned duplicated UUID %s", s)
}
m[s] = true
uuid, err := Parse(s.String())
if err != nil {
t.Errorf("New.String() returned %q which does not decode", s)
continue
}
if v := uuid.Version(); v != 4 {
t.Errorf("Random UUID of version %s", v)
}
if uuid.Variant() != RFC4122 {
t.Errorf("Random UUID is variant %d", uuid.Variant())
}
}
}
func TestClockSeq(t *testing.T) {
// Fake time.Now for this test to return a monotonically advancing time; restore it at end.
defer func(orig func() time.Time) { timeNow = orig }(timeNow)
monTime := time.Now()
timeNow = func() time.Time {
monTime = monTime.Add(1 * time.Second)
return monTime
}
SetClockSequence(-1)
uuid1, err := NewUUID()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
}
uuid2, err := NewUUID()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
}
if s1, s2 := uuid1.ClockSequence(), uuid2.ClockSequence(); s1 != s2 {
t.Errorf("clock sequence %d != %d", s1, s2)
}
SetClockSequence(-1)
uuid2, err = NewUUID()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
}
// Just on the very off chance we generated the same sequence
// two times we try again.
if uuid1.ClockSequence() == uuid2.ClockSequence() {
SetClockSequence(-1)
uuid2, err = NewUUID()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
}
}
if s1, s2 := uuid1.ClockSequence(), uuid2.ClockSequence(); s1 == s2 {
t.Errorf("Duplicate clock sequence %d", s1)
}
SetClockSequence(0x1234)
uuid1, err = NewUUID()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
}
if seq := uuid1.ClockSequence(); seq != 0x1234 {
t.Errorf("%s: expected seq 0x1234 got 0x%04x", uuid1, seq)
}
}
func TestCoding(t *testing.T) {
text := "7d444840-9dc0-11d1-b245-5ffdce74fad2"
urn := "urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2"
data := UUID{
0x7d, 0x44, 0x48, 0x40,
0x9d, 0xc0,
0x11, 0xd1,
0xb2, 0x45,
0x5f, 0xfd, 0xce, 0x74, 0xfa, 0xd2,
}
if v := data.String(); v != text {
t.Errorf("%x: encoded to %s, expected %s", data, v, text)
}
if v := data.URN(); v != urn {
t.Errorf("%x: urn is %s, expected %s", data, v, urn)
}
uuid, err := Parse(text)
if err != nil {
t.Errorf("Parse returned unexpected error %v", err)
}
if data != uuid {
t.Errorf("%s: decoded to %s, expected %s", text, uuid, data)
}
}
func TestVersion1(t *testing.T) {
uuid1, err := NewUUID()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
}
uuid2, err := NewUUID()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
}
if uuid1 == uuid2 {
t.Errorf("%s:duplicate uuid", uuid1)
}
if v := uuid1.Version(); v != 1 {
t.Errorf("%s: version %s expected 1", uuid1, v)
}
if v := uuid2.Version(); v != 1 {
t.Errorf("%s: version %s expected 1", uuid2, v)
}
n1 := uuid1.NodeID()
n2 := uuid2.NodeID()
if !bytes.Equal(n1, n2) {
t.Errorf("Different nodes %x != %x", n1, n2)
}
t1 := uuid1.Time()
t2 := uuid2.Time()
q1 := uuid1.ClockSequence()
q2 := uuid2.ClockSequence()
switch {
case t1 == t2 && q1 == q2:
t.Error("time stopped")
case t1 > t2 && q1 == q2:
t.Error("time reversed")
case t1 < t2 && q1 != q2:
t.Error("clock sequence chaned unexpectedly")
}
}
func TestNode(t *testing.T) {
// This test is mostly to make sure we don't leave nodeMu locked.
ifname = ""
if ni := NodeInterface(); ni != "" {
t.Errorf("NodeInterface got %q, want %q", ni, "")
}
if SetNodeInterface("xyzzy") {
t.Error("SetNodeInterface succeeded on a bad interface name")
}
if !SetNodeInterface("") {
t.Error("SetNodeInterface failed")
}
if ni := NodeInterface(); ni == "" {
t.Error("NodeInterface returned an empty string")
}
ni := NodeID()
if len(ni) != 6 {
t.Errorf("ni got %d bytes, want 6", len(ni))
}
hasData := false
for _, b := range ni {
if b != 0 {
hasData = true
}
}
if !hasData {
t.Error("nodeid is all zeros")
}
id := []byte{1, 2, 3, 4, 5, 6, 7, 8}
SetNodeID(id)
ni = NodeID()
if !bytes.Equal(ni, id[:6]) {
t.Errorf("got nodeid %v, want %v", ni, id[:6])
}
if ni := NodeInterface(); ni != "user" {
t.Errorf("got inteface %q, want %q", ni, "user")
}
}
func TestNodeAndTime(t *testing.T) {
// Time is February 5, 1998 12:30:23.136364800 AM GMT
uuid, err := Parse("7d444840-9dc0-11d1-b245-5ffdce74fad2")
if err != nil {
t.Fatalf("Parser returned unexpected error %v", err)
}
node := []byte{0x5f, 0xfd, 0xce, 0x74, 0xfa, 0xd2}
ts := uuid.Time()
c := time.Unix(ts.UnixTime())
want := time.Date(1998, 2, 5, 0, 30, 23, 136364800, time.UTC)
if !c.Equal(want) {
t.Errorf("Got time %v, want %v", c, want)
}
if !bytes.Equal(node, uuid.NodeID()) {
t.Errorf("Expected node %v got %v", node, uuid.NodeID())
}
}
func TestMD5(t *testing.T) {
uuid := NewMD5(NameSpaceDNS, []byte("python.org")).String()
want := "6fa459ea-ee8a-3ca4-894e-db77e160355e"
if uuid != want {
t.Errorf("MD5: got %q expected %q", uuid, want)
}
}
func TestSHA1(t *testing.T) {
uuid := NewSHA1(NameSpaceDNS, []byte("python.org")).String()
want := "886313e1-3b8a-5372-9b90-0c9aee199e5d"
if uuid != want {
t.Errorf("SHA1: got %q expected %q", uuid, want)
}
}
func TestNodeID(t *testing.T) {
nid := []byte{1, 2, 3, 4, 5, 6}
SetNodeInterface("")
s := NodeInterface()
if s == "" || s == "user" {
t.Errorf("NodeInterface %q after SetInteface", s)
}
node1 := NodeID()
if node1 == nil {
t.Error("NodeID nil after SetNodeInterface", s)
}
SetNodeID(nid)
s = NodeInterface()
if s != "user" {
t.Errorf("Expected NodeInterface %q got %q", "user", s)
}
node2 := NodeID()
if node2 == nil {
t.Error("NodeID nil after SetNodeID", s)
}
if bytes.Equal(node1, node2) {
t.Error("NodeID not changed after SetNodeID", s)
} else if !bytes.Equal(nid, node2) {
t.Errorf("NodeID is %x, expected %x", node2, nid)
}
}
func testDCE(t *testing.T, name string, uuid UUID, err error, domain Domain, id uint32) {
if err != nil {
t.Errorf("%s failed: %v", name, err)
return
}
if v := uuid.Version(); v != 2 {
t.Errorf("%s: %s: expected version 2, got %s", name, uuid, v)
return
}
if v := uuid.Domain(); v != domain {
t.Errorf("%s: %s: expected domain %d, got %d", name, uuid, domain, v)
}
if v := uuid.ID(); v != id {
t.Errorf("%s: %s: expected id %d, got %d", name, uuid, id, v)
}
}
func TestDCE(t *testing.T) {
uuid, err := NewDCESecurity(42, 12345678)
testDCE(t, "NewDCESecurity", uuid, err, 42, 12345678)
uuid, err = NewDCEPerson()
testDCE(t, "NewDCEPerson", uuid, err, Person, uint32(os.Getuid()))
uuid, err = NewDCEGroup()
testDCE(t, "NewDCEGroup", uuid, err, Group, uint32(os.Getgid()))
}
type badRand struct{}
func (r badRand) Read(buf []byte) (int, error) {
for i, _ := range buf {
buf[i] = byte(i)
}
return len(buf), nil
}
func TestBadRand(t *testing.T) {
SetRand(badRand{})
uuid1 := New()
uuid2 := New()
if uuid1 != uuid2 {
t.Errorf("execpted duplicates, got %q and %q", uuid1, uuid2)
}
SetRand(nil)
uuid1 = New()
uuid2 = New()
if uuid1 == uuid2 {
t.Errorf("unexecpted duplicates, got %q", uuid1)
}
}
var asString = "f47ac10b-58cc-0372-8567-0e02b2c3d479"
var asBytes = []byte(asString)
func BenchmarkParse(b *testing.B) {
for i := 0; i < b.N; i++ {
_, err := Parse(asString)
if err != nil {
b.Fatal(err)
}
}
}
func BenchmarkParseBytes(b *testing.B) {
for i := 0; i < b.N; i++ {
_, err := ParseBytes(asBytes)
if err != nil {
b.Fatal(err)
}
}
}
// parseBytesUnsafe is to benchmark using unsafe.
func parseBytesUnsafe(b []byte) (UUID, error) {
return Parse(*(*string)(unsafe.Pointer(&b)))
}
func BenchmarkParseBytesUnsafe(b *testing.B) {
for i := 0; i < b.N; i++ {
_, err := parseBytesUnsafe(asBytes)
if err != nil {
b.Fatal(err)
}
}
}
// parseBytesCopy is to benchmark not using unsafe.
func parseBytesCopy(b []byte) (UUID, error) {
return Parse(string(b))
}
func BenchmarkParseBytesCopy(b *testing.B) {
for i := 0; i < b.N; i++ {
_, err := parseBytesCopy(asBytes)
if err != nil {
b.Fatal(err)
}
}
}
func BenchmarkNew(b *testing.B) {
for i := 0; i < b.N; i++ {
New()
}
}
func BenchmarkUUID_String(b *testing.B) {
uuid, err := Parse("f47ac10b-58cc-0372-8567-0e02b2c3d479")
if err != nil {
b.Fatal(err)
}
for i := 0; i < b.N; i++ {
if uuid.String() == "" {
b.Fatal("invalid uuid")
}
}
}
func BenchmarkUUID_URN(b *testing.B) {
uuid, err := Parse("f47ac10b-58cc-0372-8567-0e02b2c3d479")
if err != nil {
b.Fatal(err)
}
for i := 0; i < b.N; i++ {
if uuid.URN() == "" {
b.Fatal("invalid uuid")
}
}
}

View file

@ -13,7 +13,7 @@ import (
// or SetNodeInterface then it will be set automatically. If the NodeID cannot
// be set NewUUID returns nil. If clock sequence has not been set by
// SetClockSequence then it will be set automatically. If GetTime fails to
// return the current NewUUID returns Nil and an error.
// return the current NewUUID returns nil and an error.
//
// In most cases, New should be used.
func NewUUID() (UUID, error) {

View file

@ -14,12 +14,12 @@ func New() UUID {
return Must(NewRandom())
}
// NewRandom returns a Random (Version 4) UUID or panics.
// NewRandom returns a Random (Version 4) UUID.
//
// The strength of the UUIDs is based on the strength of the crypto/rand
// package.
//
// A note about uniqueness derived from from the UUID Wikipedia entry:
// A note about uniqueness derived from the UUID Wikipedia entry:
//
// Randomly generated UUIDs have 122 random bits. One's annual risk of being
// hit by a meteorite is estimated to be one chance in 17 billion, that

View file

@ -35,7 +35,7 @@ buildfuzz:
go-fuzz-build github.com/jmespath/go-jmespath/fuzz
fuzz: buildfuzz
go-fuzz -bin=./jmespath-fuzz.zip -workdir=fuzz/corpus
go-fuzz -bin=./jmespath-fuzz.zip -workdir=fuzz/testdata
bench:
go test -bench . -cpuprofile cpu.out

View file

@ -1,5 +1,42 @@
package jmespath
import "strconv"
// JMESPath is the epresentation of a compiled JMES path query. A JMESPath is
// safe for concurrent use by multiple goroutines.
type JMESPath struct {
ast ASTNode
intr *treeInterpreter
}
// Compile parses a JMESPath expression and returns, if successful, a JMESPath
// object that can be used to match against data.
func Compile(expression string) (*JMESPath, error) {
parser := NewParser()
ast, err := parser.Parse(expression)
if err != nil {
return nil, err
}
jmespath := &JMESPath{ast: ast, intr: newInterpreter()}
return jmespath, nil
}
// MustCompile is like Compile but panics if the expression cannot be parsed.
// It simplifies safe initialization of global variables holding compiled
// JMESPaths.
func MustCompile(expression string) *JMESPath {
jmespath, err := Compile(expression)
if err != nil {
panic(`jmespath: Compile(` + strconv.Quote(expression) + `): ` + err.Error())
}
return jmespath
}
// Search evaluates a JMESPath expression against input data and returns the result.
func (jp *JMESPath) Search(data interface{}) (interface{}, error) {
return jp.intr.Execute(jp.ast, data)
}
// Search evaluates a JMESPath expression against input data and returns the result.
func Search(expression string, data interface{}) (interface{}, error) {
intr := newInterpreter()

View file

@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"math"
"reflect"
"sort"
"strconv"
"strings"
@ -124,197 +125,197 @@ type functionCaller struct {
func newFunctionCaller() *functionCaller {
caller := &functionCaller{}
caller.functionTable = map[string]functionEntry{
"length": functionEntry{
"length": {
name: "length",
arguments: []argSpec{
argSpec{types: []jpType{jpString, jpArray, jpObject}},
{types: []jpType{jpString, jpArray, jpObject}},
},
handler: jpfLength,
},
"starts_with": functionEntry{
"starts_with": {
name: "starts_with",
arguments: []argSpec{
argSpec{types: []jpType{jpString}},
argSpec{types: []jpType{jpString}},
{types: []jpType{jpString}},
{types: []jpType{jpString}},
},
handler: jpfStartsWith,
},
"abs": functionEntry{
"abs": {
name: "abs",
arguments: []argSpec{
argSpec{types: []jpType{jpNumber}},
{types: []jpType{jpNumber}},
},
handler: jpfAbs,
},
"avg": functionEntry{
"avg": {
name: "avg",
arguments: []argSpec{
argSpec{types: []jpType{jpArrayNumber}},
{types: []jpType{jpArrayNumber}},
},
handler: jpfAvg,
},
"ceil": functionEntry{
"ceil": {
name: "ceil",
arguments: []argSpec{
argSpec{types: []jpType{jpNumber}},
{types: []jpType{jpNumber}},
},
handler: jpfCeil,
},
"contains": functionEntry{
"contains": {
name: "contains",
arguments: []argSpec{
argSpec{types: []jpType{jpArray, jpString}},
argSpec{types: []jpType{jpAny}},
{types: []jpType{jpArray, jpString}},
{types: []jpType{jpAny}},
},
handler: jpfContains,
},
"ends_with": functionEntry{
"ends_with": {
name: "ends_with",
arguments: []argSpec{
argSpec{types: []jpType{jpString}},
argSpec{types: []jpType{jpString}},
{types: []jpType{jpString}},
{types: []jpType{jpString}},
},
handler: jpfEndsWith,
},
"floor": functionEntry{
"floor": {
name: "floor",
arguments: []argSpec{
argSpec{types: []jpType{jpNumber}},
{types: []jpType{jpNumber}},
},
handler: jpfFloor,
},
"map": functionEntry{
"map": {
name: "amp",
arguments: []argSpec{
argSpec{types: []jpType{jpExpref}},
argSpec{types: []jpType{jpArray}},
{types: []jpType{jpExpref}},
{types: []jpType{jpArray}},
},
handler: jpfMap,
hasExpRef: true,
},
"max": functionEntry{
"max": {
name: "max",
arguments: []argSpec{
argSpec{types: []jpType{jpArrayNumber, jpArrayString}},
{types: []jpType{jpArrayNumber, jpArrayString}},
},
handler: jpfMax,
},
"merge": functionEntry{
"merge": {
name: "merge",
arguments: []argSpec{
argSpec{types: []jpType{jpObject}, variadic: true},
{types: []jpType{jpObject}, variadic: true},
},
handler: jpfMerge,
},
"max_by": functionEntry{
"max_by": {
name: "max_by",
arguments: []argSpec{
argSpec{types: []jpType{jpArray}},
argSpec{types: []jpType{jpExpref}},
{types: []jpType{jpArray}},
{types: []jpType{jpExpref}},
},
handler: jpfMaxBy,
hasExpRef: true,
},
"sum": functionEntry{
"sum": {
name: "sum",
arguments: []argSpec{
argSpec{types: []jpType{jpArrayNumber}},
{types: []jpType{jpArrayNumber}},
},
handler: jpfSum,
},
"min": functionEntry{
"min": {
name: "min",
arguments: []argSpec{
argSpec{types: []jpType{jpArrayNumber, jpArrayString}},
{types: []jpType{jpArrayNumber, jpArrayString}},
},
handler: jpfMin,
},
"min_by": functionEntry{
"min_by": {
name: "min_by",
arguments: []argSpec{
argSpec{types: []jpType{jpArray}},
argSpec{types: []jpType{jpExpref}},
{types: []jpType{jpArray}},
{types: []jpType{jpExpref}},
},
handler: jpfMinBy,
hasExpRef: true,
},
"type": functionEntry{
"type": {
name: "type",
arguments: []argSpec{
argSpec{types: []jpType{jpAny}},
{types: []jpType{jpAny}},
},
handler: jpfType,
},
"keys": functionEntry{
"keys": {
name: "keys",
arguments: []argSpec{
argSpec{types: []jpType{jpObject}},
{types: []jpType{jpObject}},
},
handler: jpfKeys,
},
"values": functionEntry{
"values": {
name: "values",
arguments: []argSpec{
argSpec{types: []jpType{jpObject}},
{types: []jpType{jpObject}},
},
handler: jpfValues,
},
"sort": functionEntry{
"sort": {
name: "sort",
arguments: []argSpec{
argSpec{types: []jpType{jpArrayString, jpArrayNumber}},
{types: []jpType{jpArrayString, jpArrayNumber}},
},
handler: jpfSort,
},
"sort_by": functionEntry{
"sort_by": {
name: "sort_by",
arguments: []argSpec{
argSpec{types: []jpType{jpArray}},
argSpec{types: []jpType{jpExpref}},
{types: []jpType{jpArray}},
{types: []jpType{jpExpref}},
},
handler: jpfSortBy,
hasExpRef: true,
},
"join": functionEntry{
"join": {
name: "join",
arguments: []argSpec{
argSpec{types: []jpType{jpString}},
argSpec{types: []jpType{jpArrayString}},
{types: []jpType{jpString}},
{types: []jpType{jpArrayString}},
},
handler: jpfJoin,
},
"reverse": functionEntry{
"reverse": {
name: "reverse",
arguments: []argSpec{
argSpec{types: []jpType{jpArray, jpString}},
{types: []jpType{jpArray, jpString}},
},
handler: jpfReverse,
},
"to_array": functionEntry{
"to_array": {
name: "to_array",
arguments: []argSpec{
argSpec{types: []jpType{jpAny}},
{types: []jpType{jpAny}},
},
handler: jpfToArray,
},
"to_string": functionEntry{
"to_string": {
name: "to_string",
arguments: []argSpec{
argSpec{types: []jpType{jpAny}},
{types: []jpType{jpAny}},
},
handler: jpfToString,
},
"to_number": functionEntry{
"to_number": {
name: "to_number",
arguments: []argSpec{
argSpec{types: []jpType{jpAny}},
{types: []jpType{jpAny}},
},
handler: jpfToNumber,
},
"not_null": functionEntry{
"not_null": {
name: "not_null",
arguments: []argSpec{
argSpec{types: []jpType{jpAny}, variadic: true},
{types: []jpType{jpAny}, variadic: true},
},
handler: jpfNotNull,
},
@ -357,7 +358,7 @@ func (a *argSpec) typeCheck(arg interface{}) error {
return nil
}
case jpArray:
if _, ok := arg.([]interface{}); ok {
if isSliceType(arg) {
return nil
}
case jpObject:
@ -409,8 +410,9 @@ func jpfLength(arguments []interface{}) (interface{}, error) {
arg := arguments[0]
if c, ok := arg.(string); ok {
return float64(utf8.RuneCountInString(c)), nil
} else if c, ok := arg.([]interface{}); ok {
return float64(len(c)), nil
} else if isSliceType(arg) {
v := reflect.ValueOf(arg)
return float64(v.Len()), nil
} else if c, ok := arg.(map[string]interface{}); ok {
return float64(len(c)), nil
}

View file

@ -353,7 +353,7 @@ func (p *Parser) nud(token token) (ASTNode, error) {
case tFlatten:
left := ASTNode{
nodeType: ASTFlatten,
children: []ASTNode{ASTNode{nodeType: ASTIdentity}},
children: []ASTNode{{nodeType: ASTIdentity}},
}
right, err := p.parseProjectionRHS(bindingPowers[tFlatten])
if err != nil {
@ -378,7 +378,7 @@ func (p *Parser) nud(token token) (ASTNode, error) {
}
return ASTNode{
nodeType: ASTProjection,
children: []ASTNode{ASTNode{nodeType: ASTIdentity}, right},
children: []ASTNode{{nodeType: ASTIdentity}, right},
}, nil
} else {
return p.parseMultiSelectList()

View file

@ -11,7 +11,7 @@ import (
"github.com/miekg/dns"
)
// AddDomain adds origin to s if s is not already a FQDN.
// AddOrigin adds origin to s if s is not already a FQDN.
// Note that the result may not be a FQDN. If origin does not end
// with a ".", the result won't either.
// This implements the zonefile convention (specified in RFC 1035,
@ -20,7 +20,9 @@ import (
func AddOrigin(s, origin string) string {
// ("foo.", "origin.") -> "foo." (already a FQDN)
// ("foo", "origin.") -> "foo.origin."
// ("foo"), "origin" -> "foo.origin"
// ("foo", "origin") -> "foo.origin"
// ("foo", ".") -> "foo." (Same as dns.Fqdn())
// ("foo.", ".") -> "foo." (Same as dns.Fqdn())
// ("@", "origin.") -> "origin." (@ represents the apex (bare) domain)
// ("", "origin.") -> "origin." (not obvious)
// ("foo", "") -> "foo" (not obvious)
@ -34,32 +36,34 @@ func AddOrigin(s, origin string) string {
if s == "@" || len(s) == 0 {
return origin // Expand apex.
}
if origin == "." {
return s + origin // AddOrigin(s, ".") is an expensive way to add a ".".
return dns.Fqdn(s)
}
return s + "." + origin // The simple case.
}
// TrimDomainName trims origin from s if s is a subdomain.
// This function will never return "", but returns "@" instead (@ represents the apex (bare) domain).
// This function will never return "", but returns "@" instead (@ represents the apex domain).
func TrimDomainName(s, origin string) string {
// An apex (bare) domain is always returned as "@".
// If the return value ends in a ".", the domain was not the suffix.
// origin can end in "." or not. Either way the results should be the same.
if len(s) == 0 {
return "@" // Return the apex (@) rather than "".
return "@"
}
// Someone is using TrimDomainName(s, ".") to remove a dot if it exists.
if origin == "." {
return strings.TrimSuffix(s, origin)
}
// Dude, you aren't even if the right subdomain!
original := s
s = dns.Fqdn(s)
origin = dns.Fqdn(origin)
if !dns.IsSubDomain(origin, s) {
return s
return original
}
slabels := dns.Split(s)

View file

@ -79,14 +79,6 @@ func (f Frame) Format(s fmt.State, verb rune) {
// StackTrace is stack of Frames from innermost (newest) to outermost (oldest).
type StackTrace []Frame
// Format formats the stack of Frames according to the fmt.Formatter interface.
//
// %s lists source files for each Frame in the stack
// %v lists the source file and line number for each Frame in the stack
//
// Format accepts flags that alter the printing of some verbs, as follows:
//
// %+v Prints filename, function, and line number for each Frame in the stack.
func (st StackTrace) Format(s fmt.State, verb rune) {
switch verb {
case 'v':

View file

@ -32,6 +32,7 @@ applications in an expressive way.
+ [Alternate Names](#alternate-names)
+ [Ordering](#ordering)
+ [Values from the Environment](#values-from-the-environment)
+ [Values from files](#values-from-files)
+ [Values from alternate input sources (YAML, TOML, and others)](#values-from-alternate-input-sources-yaml-toml-and-others)
+ [Precedence](#precedence)
* [Subcommands](#subcommands)
@ -46,6 +47,7 @@ applications in an expressive way.
* [Version Flag](#version-flag)
+ [Customization](#customization-2)
+ [Full API Example](#full-api-example)
* [Combining short Bool options](#combining-short-bool-options)
- [Contribution Guidelines](#contribution-guidelines)
<!-- tocstop -->
@ -587,6 +589,41 @@ func main() {
}
```
#### Values from files
You can also have the default value set from file via `FilePath`. e.g.
<!-- {
"args": ["&#45;&#45;help"],
"output": "password for the mysql database"
} -->
``` go
package main
import (
"os"
"github.com/urfave/cli"
)
func main() {
app := cli.NewApp()
app.Flags = []cli.Flag {
cli.StringFlag{
Name: "password, p",
Usage: "password for the mysql database",
FilePath: "/etc/mysql/password",
},
}
app.Run(os.Args)
}
```
Note that default values set from file (e.g. `FilePath`) take precedence over
default values set from the enviornment (e.g. `EnvVar`).
#### Values from alternate input sources (YAML, TOML, and others)
There is a separate package altsrc that adds support for getting flag values
@ -1374,6 +1411,26 @@ func wopAction(c *cli.Context) error {
}
```
### Combining short Bool options
Traditional use of boolean options using their shortnames look like this:
```
# cmd foobar -s -o
```
Suppose you want users to be able to combine your bool options with their shortname. This
can be done using the **UseShortOptionHandling** bool in your commands. Suppose your program
has a two bool flags such as *serve* and *option* with the short options of *-o* and
*-s* respectively. With **UseShortOptionHandling** set to *true*, a user can use a syntax
like:
```
# cmd foobar -so
```
If you enable the **UseShortOptionHandling*, then you must not use any flags that have a single
leading *-* or this will result in failures. For example, **-option** can no longer be used. Flags
with two leading dashes (such as **--options**) are still valid.
## Contribution Guidelines
Feel free to put up a pull request to fix a bug or maybe add a feature. I will

31
vendor/github.com/urfave/cli/app.go generated vendored
View file

@ -83,6 +83,9 @@ type App struct {
Writer io.Writer
// ErrWriter writes error output
ErrWriter io.Writer
// Execute this function to handle ExitErrors. If not provided, HandleExitCoder is provided to
// function as a default, so this is optional.
ExitErrHandler ExitErrHandlerFunc
// Other custom info
Metadata map[string]interface{}
// Carries a function which returns app specific info.
@ -207,7 +210,7 @@ func (a *App) Run(arguments []string) (err error) {
if err != nil {
if a.OnUsageError != nil {
err := a.OnUsageError(context, err, false)
HandleExitCoder(err)
a.handleExitCoder(context, err)
return err
}
fmt.Fprintf(a.Writer, "%s %s\n\n", "Incorrect Usage.", err.Error())
@ -240,8 +243,9 @@ func (a *App) Run(arguments []string) (err error) {
if a.Before != nil {
beforeErr := a.Before(context)
if beforeErr != nil {
fmt.Fprintf(a.Writer, "%v\n\n", beforeErr)
ShowAppHelp(context)
HandleExitCoder(beforeErr)
a.handleExitCoder(context, beforeErr)
err = beforeErr
return err
}
@ -263,7 +267,7 @@ func (a *App) Run(arguments []string) (err error) {
// Run default Action
err = HandleAction(a.Action, context)
HandleExitCoder(err)
a.handleExitCoder(context, err)
return err
}
@ -330,7 +334,7 @@ func (a *App) RunAsSubcommand(ctx *Context) (err error) {
if err != nil {
if a.OnUsageError != nil {
err = a.OnUsageError(context, err, true)
HandleExitCoder(err)
a.handleExitCoder(context, err)
return err
}
fmt.Fprintf(a.Writer, "%s %s\n\n", "Incorrect Usage.", err.Error())
@ -352,7 +356,7 @@ func (a *App) RunAsSubcommand(ctx *Context) (err error) {
defer func() {
afterErr := a.After(context)
if afterErr != nil {
HandleExitCoder(err)
a.handleExitCoder(context, err)
if err != nil {
err = NewMultiError(err, afterErr)
} else {
@ -365,7 +369,7 @@ func (a *App) RunAsSubcommand(ctx *Context) (err error) {
if a.Before != nil {
beforeErr := a.Before(context)
if beforeErr != nil {
HandleExitCoder(beforeErr)
a.handleExitCoder(context, beforeErr)
err = beforeErr
return err
}
@ -383,7 +387,7 @@ func (a *App) RunAsSubcommand(ctx *Context) (err error) {
// Run default Action
err = HandleAction(a.Action, context)
HandleExitCoder(err)
a.handleExitCoder(context, err)
return err
}
@ -449,7 +453,6 @@ func (a *App) hasFlag(flag Flag) bool {
}
func (a *App) errWriter() io.Writer {
// When the app ErrWriter is nil use the package level one.
if a.ErrWriter == nil {
return ErrWriter
@ -464,6 +467,14 @@ func (a *App) appendFlag(flag Flag) {
}
}
func (a *App) handleExitCoder(context *Context, err error) {
if a.ExitErrHandler != nil {
a.ExitErrHandler(context, err)
} else {
HandleExitCoder(err)
}
}
// Author represents someone who has contributed to a cli project.
type Author struct {
Name string // The Authors name
@ -491,7 +502,7 @@ func HandleAction(action interface{}, context *Context) (err error) {
} else if a, ok := action.(func(*Context)); ok { // deprecated function signature
a(context)
return nil
} else {
return errInvalidActionType
}
return errInvalidActionType
}

View file

@ -10,7 +10,7 @@ type CommandCategory struct {
}
func (c CommandCategories) Less(i, j int) bool {
return c[i].Name < c[j].Name
return lexicographicLess(c[i].Name, c[j].Name)
}
func (c CommandCategories) Len() int {

View file

@ -1,6 +1,7 @@
package cli
import (
"flag"
"fmt"
"io/ioutil"
"sort"
@ -55,6 +56,10 @@ type Command struct {
HideHelp bool
// Boolean to hide this command from help or completion
Hidden bool
// Boolean to enable short-option handling so user can combine several
// single-character bool arguements into one
// i.e. foobar -o -v -> foobar -ov
UseShortOptionHandling bool
// Full name of command for help, defaults to full command name, including parent commands.
HelpName string
@ -73,7 +78,7 @@ func (c CommandsByName) Len() int {
}
func (c CommandsByName) Less(i, j int) bool {
return c[i].Name < c[j].Name
return lexicographicLess(c[i].Name, c[j].Name)
}
func (c CommandsByName) Swap(i, j int) {
@ -106,57 +111,7 @@ func (c Command) Run(ctx *Context) (err error) {
)
}
set, err := flagSet(c.Name, c.Flags)
if err != nil {
return err
}
set.SetOutput(ioutil.Discard)
if c.SkipFlagParsing {
err = set.Parse(append([]string{"--"}, ctx.Args().Tail()...))
} else if !c.SkipArgReorder {
firstFlagIndex := -1
terminatorIndex := -1
for index, arg := range ctx.Args() {
if arg == "--" {
terminatorIndex = index
break
} else if arg == "-" {
// Do nothing. A dash alone is not really a flag.
continue
} else if strings.HasPrefix(arg, "-") && firstFlagIndex == -1 {
firstFlagIndex = index
}
}
if firstFlagIndex > -1 {
args := ctx.Args()
regularArgs := make([]string, len(args[1:firstFlagIndex]))
copy(regularArgs, args[1:firstFlagIndex])
var flagArgs []string
if terminatorIndex > -1 {
flagArgs = args[firstFlagIndex:terminatorIndex]
regularArgs = append(regularArgs, args[terminatorIndex:]...)
} else {
flagArgs = args[firstFlagIndex:]
}
err = set.Parse(append(flagArgs, regularArgs...))
} else {
err = set.Parse(ctx.Args().Tail())
}
} else {
err = set.Parse(ctx.Args().Tail())
}
nerr := normalizeFlags(c.Flags, set)
if nerr != nil {
fmt.Fprintln(ctx.App.Writer, nerr)
fmt.Fprintln(ctx.App.Writer)
ShowCommandHelp(ctx, c.Name)
return nerr
}
set, err := c.parseFlags(ctx.Args().Tail())
context := NewContext(ctx.App, set, ctx)
context.Command = c
@ -167,7 +122,7 @@ func (c Command) Run(ctx *Context) (err error) {
if err != nil {
if c.OnUsageError != nil {
err := c.OnUsageError(context, err, false)
HandleExitCoder(err)
context.App.handleExitCoder(context, err)
return err
}
fmt.Fprintln(context.App.Writer, "Incorrect Usage:", err.Error())
@ -184,7 +139,7 @@ func (c Command) Run(ctx *Context) (err error) {
defer func() {
afterErr := c.After(context)
if afterErr != nil {
HandleExitCoder(err)
context.App.handleExitCoder(context, err)
if err != nil {
err = NewMultiError(err, afterErr)
} else {
@ -198,7 +153,7 @@ func (c Command) Run(ctx *Context) (err error) {
err = c.Before(context)
if err != nil {
ShowCommandHelp(context, c.Name)
HandleExitCoder(err)
context.App.handleExitCoder(context, err)
return err
}
}
@ -210,11 +165,87 @@ func (c Command) Run(ctx *Context) (err error) {
err = HandleAction(c.Action, context)
if err != nil {
HandleExitCoder(err)
context.App.handleExitCoder(context, err)
}
return err
}
func (c *Command) parseFlags(args Args) (*flag.FlagSet, error) {
set, err := flagSet(c.Name, c.Flags)
if err != nil {
return nil, err
}
set.SetOutput(ioutil.Discard)
if c.SkipFlagParsing {
return set, set.Parse(append([]string{"--"}, args...))
}
if c.UseShortOptionHandling {
args = translateShortOptions(args)
}
if !c.SkipArgReorder {
args = reorderArgs(args)
}
err = set.Parse(args)
if err != nil {
return nil, err
}
err = normalizeFlags(c.Flags, set)
if err != nil {
return nil, err
}
return set, nil
}
// reorderArgs moves all flags before arguments as this is what flag expects
func reorderArgs(args []string) []string {
var nonflags, flags []string
readFlagValue := false
for i, arg := range args {
if arg == "--" {
nonflags = append(nonflags, args[i:]...)
break
}
if readFlagValue {
readFlagValue = false
flags = append(flags, arg)
continue
}
if arg != "-" && strings.HasPrefix(arg, "-") {
flags = append(flags, arg)
readFlagValue = !strings.Contains(arg, "=")
} else {
nonflags = append(nonflags, arg)
}
}
return append(flags, nonflags...)
}
func translateShortOptions(flagArgs Args) []string {
// separate combined flags
var flagArgsSeparated []string
for _, flagArg := range flagArgs {
if strings.HasPrefix(flagArg, "-") && strings.HasPrefix(flagArg, "--") == false && len(flagArg) > 2 {
for _, flagChar := range flagArg[1:] {
flagArgsSeparated = append(flagArgsSeparated, "-"+string(flagChar))
}
} else {
flagArgsSeparated = append(flagArgsSeparated, flagArg)
}
}
return flagArgsSeparated
}
// Names returns the names including short names and aliases.
func (c Command) Names() []string {
names := []string{c.Name}

View file

@ -3,6 +3,7 @@ package cli
import (
"errors"
"flag"
"os"
"reflect"
"strings"
"syscall"
@ -73,7 +74,7 @@ func (c *Context) IsSet(name string) bool {
// change in version 2 to add `IsSet` to the Flag interface to push the
// responsibility closer to where the information required to determine
// whether a flag is set by non-standard means such as environment
// variables is avaliable.
// variables is available.
//
// See https://github.com/urfave/cli/issues/294 for additional discussion
flags := c.Command.Flags
@ -93,18 +94,26 @@ func (c *Context) IsSet(name string) bool {
val = val.Elem()
}
envVarValue := val.FieldByName("EnvVar")
if !envVarValue.IsValid() {
return
filePathValue := val.FieldByName("FilePath")
if filePathValue.IsValid() {
eachName(filePathValue.String(), func(filePath string) {
if _, err := os.Stat(filePath); err == nil {
c.setFlags[name] = true
return
}
})
}
eachName(envVarValue.String(), func(envVar string) {
envVar = strings.TrimSpace(envVar)
if _, ok := syscall.Getenv(envVar); ok {
c.setFlags[name] = true
return
}
})
envVarValue := val.FieldByName("EnvVar")
if envVarValue.IsValid() {
eachName(envVarValue.String(), func(envVar string) {
envVar = strings.TrimSpace(envVar)
if _, ok := syscall.Getenv(envVar); ok {
c.setFlags[name] = true
return
}
})
}
})
}
}

323
vendor/github.com/urfave/cli/flag.go generated vendored
View file

@ -3,6 +3,7 @@ package cli
import (
"flag"
"fmt"
"io/ioutil"
"reflect"
"runtime"
"strconv"
@ -37,6 +38,18 @@ var HelpFlag Flag = BoolFlag{
// to display a flag.
var FlagStringer FlagStringFunc = stringifyFlag
// FlagNamePrefixer converts a full flag name and its placeholder into the help
// message flag prefix. This is used by the default FlagStringer.
var FlagNamePrefixer FlagNamePrefixFunc = prefixedNames
// FlagEnvHinter annotates flag help message with the environment variable
// details. This is used by the default FlagStringer.
var FlagEnvHinter FlagEnvHintFunc = withEnvHint
// FlagFileHinter annotates flag help message with the environment variable
// details. This is used by the default FlagStringer.
var FlagFileHinter FlagFileHintFunc = withFileHint
// FlagsByName is a slice of Flag.
type FlagsByName []Flag
@ -45,7 +58,7 @@ func (f FlagsByName) Len() int {
}
func (f FlagsByName) Less(i, j int) bool {
return f[i].GetName() < f[j].GetName()
return lexicographicLess(f[i].GetName(), f[j].GetName())
}
func (f FlagsByName) Swap(i, j int) {
@ -112,15 +125,9 @@ func (f GenericFlag) Apply(set *flag.FlagSet) {
// provided by the user for parsing by the flag
func (f GenericFlag) ApplyWithError(set *flag.FlagSet) error {
val := f.Value
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
if err := val.Set(envVal); err != nil {
return fmt.Errorf("could not parse %s as value for flag %s: %s", envVal, f.Name, err)
}
break
}
if fileEnvVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
if err := val.Set(fileEnvVal); err != nil {
return fmt.Errorf("could not parse %s as value for flag %s: %s", fileEnvVal, f.Name, err)
}
}
@ -163,21 +170,19 @@ func (f StringSliceFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f StringSliceFlag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
newVal := &StringSlice{}
for _, s := range strings.Split(envVal, ",") {
s = strings.TrimSpace(s)
if err := newVal.Set(s); err != nil {
return fmt.Errorf("could not parse %s as string value for flag %s: %s", envVal, f.Name, err)
}
}
f.Value = newVal
break
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
newVal := &StringSlice{}
for _, s := range strings.Split(envVal, ",") {
s = strings.TrimSpace(s)
if err := newVal.Set(s); err != nil {
return fmt.Errorf("could not parse %s as string value for flag %s: %s", envVal, f.Name, err)
}
}
if f.Value == nil {
f.Value = newVal
} else {
*f.Value = *newVal
}
}
eachName(f.Name, func(name string) {
@ -226,21 +231,19 @@ func (f IntSliceFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f IntSliceFlag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
newVal := &IntSlice{}
for _, s := range strings.Split(envVal, ",") {
s = strings.TrimSpace(s)
if err := newVal.Set(s); err != nil {
return fmt.Errorf("could not parse %s as int slice value for flag %s: %s", envVal, f.Name, err)
}
}
f.Value = newVal
break
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
newVal := &IntSlice{}
for _, s := range strings.Split(envVal, ",") {
s = strings.TrimSpace(s)
if err := newVal.Set(s); err != nil {
return fmt.Errorf("could not parse %s as int slice value for flag %s: %s", envVal, f.Name, err)
}
}
if f.Value == nil {
f.Value = newVal
} else {
*f.Value = *newVal
}
}
eachName(f.Name, func(name string) {
@ -289,21 +292,19 @@ func (f Int64SliceFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f Int64SliceFlag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
newVal := &Int64Slice{}
for _, s := range strings.Split(envVal, ",") {
s = strings.TrimSpace(s)
if err := newVal.Set(s); err != nil {
return fmt.Errorf("could not parse %s as int64 slice value for flag %s: %s", envVal, f.Name, err)
}
}
f.Value = newVal
break
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
newVal := &Int64Slice{}
for _, s := range strings.Split(envVal, ",") {
s = strings.TrimSpace(s)
if err := newVal.Set(s); err != nil {
return fmt.Errorf("could not parse %s as int64 slice value for flag %s: %s", envVal, f.Name, err)
}
}
if f.Value == nil {
f.Value = newVal
} else {
*f.Value = *newVal
}
}
eachName(f.Name, func(name string) {
@ -324,23 +325,15 @@ func (f BoolFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f BoolFlag) ApplyWithError(set *flag.FlagSet) error {
val := false
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
if envVal == "" {
val = false
break
}
envValBool, err := strconv.ParseBool(envVal)
if err != nil {
return fmt.Errorf("could not parse %s as bool value for flag %s: %s", envVal, f.Name, err)
}
val = envValBool
break
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
if envVal == "" {
val = false
} else {
envValBool, err := strconv.ParseBool(envVal)
if err != nil {
return fmt.Errorf("could not parse %s as bool value for flag %s: %s", envVal, f.Name, err)
}
val = envValBool
}
}
@ -364,23 +357,16 @@ func (f BoolTFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f BoolTFlag) ApplyWithError(set *flag.FlagSet) error {
val := true
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
if envVal == "" {
val = false
break
}
envValBool, err := strconv.ParseBool(envVal)
if err != nil {
return fmt.Errorf("could not parse %s as bool value for flag %s: %s", envVal, f.Name, err)
}
val = envValBool
break
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
if envVal == "" {
val = false
} else {
envValBool, err := strconv.ParseBool(envVal)
if err != nil {
return fmt.Errorf("could not parse %s as bool value for flag %s: %s", envVal, f.Name, err)
}
val = envValBool
}
}
@ -403,14 +389,8 @@ func (f StringFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f StringFlag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
f.Value = envVal
break
}
}
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
f.Value = envVal
}
eachName(f.Name, func(name string) {
@ -432,18 +412,12 @@ func (f IntFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f IntFlag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
envValInt, err := strconv.ParseInt(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as int value for flag %s: %s", envVal, f.Name, err)
}
f.Value = int(envValInt)
break
}
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
envValInt, err := strconv.ParseInt(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as int value for flag %s: %s", envVal, f.Name, err)
}
f.Value = int(envValInt)
}
eachName(f.Name, func(name string) {
@ -465,19 +439,13 @@ func (f Int64Flag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f Int64Flag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
envValInt, err := strconv.ParseInt(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as int value for flag %s: %s", envVal, f.Name, err)
}
f.Value = envValInt
break
}
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
envValInt, err := strconv.ParseInt(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as int value for flag %s: %s", envVal, f.Name, err)
}
f.Value = envValInt
}
eachName(f.Name, func(name string) {
@ -499,19 +467,13 @@ func (f UintFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f UintFlag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
envValInt, err := strconv.ParseUint(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as uint value for flag %s: %s", envVal, f.Name, err)
}
f.Value = uint(envValInt)
break
}
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
envValInt, err := strconv.ParseUint(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as uint value for flag %s: %s", envVal, f.Name, err)
}
f.Value = uint(envValInt)
}
eachName(f.Name, func(name string) {
@ -533,19 +495,13 @@ func (f Uint64Flag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f Uint64Flag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
envValInt, err := strconv.ParseUint(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as uint64 value for flag %s: %s", envVal, f.Name, err)
}
f.Value = uint64(envValInt)
break
}
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
envValInt, err := strconv.ParseUint(envVal, 0, 64)
if err != nil {
return fmt.Errorf("could not parse %s as uint64 value for flag %s: %s", envVal, f.Name, err)
}
f.Value = uint64(envValInt)
}
eachName(f.Name, func(name string) {
@ -567,19 +523,13 @@ func (f DurationFlag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f DurationFlag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
envValDuration, err := time.ParseDuration(envVal)
if err != nil {
return fmt.Errorf("could not parse %s as duration for flag %s: %s", envVal, f.Name, err)
}
f.Value = envValDuration
break
}
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
envValDuration, err := time.ParseDuration(envVal)
if err != nil {
return fmt.Errorf("could not parse %s as duration for flag %s: %s", envVal, f.Name, err)
}
f.Value = envValDuration
}
eachName(f.Name, func(name string) {
@ -601,19 +551,13 @@ func (f Float64Flag) Apply(set *flag.FlagSet) {
// ApplyWithError populates the flag given the flag set and environment
func (f Float64Flag) ApplyWithError(set *flag.FlagSet) error {
if f.EnvVar != "" {
for _, envVar := range strings.Split(f.EnvVar, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
envValFloat, err := strconv.ParseFloat(envVal, 10)
if err != nil {
return fmt.Errorf("could not parse %s as float64 value for flag %s: %s", envVal, f.Name, err)
}
f.Value = float64(envValFloat)
break
}
if envVal, ok := flagFromFileEnv(f.FilePath, f.EnvVar); ok {
envValFloat, err := strconv.ParseFloat(envVal, 10)
if err != nil {
return fmt.Errorf("could not parse %s as float64 value for flag %s: %s", envVal, f.Name, err)
}
f.Value = float64(envValFloat)
}
eachName(f.Name, func(name string) {
@ -697,6 +641,14 @@ func withEnvHint(envVar, str string) string {
return str + envText
}
func withFileHint(filePath, str string) string {
fileText := ""
if filePath != "" {
fileText = fmt.Sprintf(" [%s]", filePath)
}
return str + fileText
}
func flagValue(f Flag) reflect.Value {
fv := reflect.ValueOf(f)
for fv.Kind() == reflect.Ptr {
@ -710,14 +662,29 @@ func stringifyFlag(f Flag) string {
switch f.(type) {
case IntSliceFlag:
return withEnvHint(fv.FieldByName("EnvVar").String(),
stringifyIntSliceFlag(f.(IntSliceFlag)))
return FlagFileHinter(
fv.FieldByName("FilePath").String(),
FlagEnvHinter(
fv.FieldByName("EnvVar").String(),
stringifyIntSliceFlag(f.(IntSliceFlag)),
),
)
case Int64SliceFlag:
return withEnvHint(fv.FieldByName("EnvVar").String(),
stringifyInt64SliceFlag(f.(Int64SliceFlag)))
return FlagFileHinter(
fv.FieldByName("FilePath").String(),
FlagEnvHinter(
fv.FieldByName("EnvVar").String(),
stringifyInt64SliceFlag(f.(Int64SliceFlag)),
),
)
case StringSliceFlag:
return withEnvHint(fv.FieldByName("EnvVar").String(),
stringifyStringSliceFlag(f.(StringSliceFlag)))
return FlagFileHinter(
fv.FieldByName("FilePath").String(),
FlagEnvHinter(
fv.FieldByName("EnvVar").String(),
stringifyStringSliceFlag(f.(StringSliceFlag)),
),
)
}
placeholder, usage := unquoteUsage(fv.FieldByName("Usage").String())
@ -744,8 +711,13 @@ func stringifyFlag(f Flag) string {
usageWithDefault := strings.TrimSpace(fmt.Sprintf("%s%s", usage, defaultValueString))
return withEnvHint(fv.FieldByName("EnvVar").String(),
fmt.Sprintf("%s\t%s", prefixedNames(fv.FieldByName("Name").String(), placeholder), usageWithDefault))
return FlagFileHinter(
fv.FieldByName("FilePath").String(),
FlagEnvHinter(
fv.FieldByName("EnvVar").String(),
fmt.Sprintf("%s\t%s", FlagNamePrefixer(fv.FieldByName("Name").String(), placeholder), usageWithDefault),
),
)
}
func stringifyIntSliceFlag(f IntSliceFlag) string {
@ -795,5 +767,20 @@ func stringifySliceFlag(usage, name string, defaultVals []string) string {
}
usageWithDefault := strings.TrimSpace(fmt.Sprintf("%s%s", usage, defaultVal))
return fmt.Sprintf("%s\t%s", prefixedNames(name, placeholder), usageWithDefault)
return fmt.Sprintf("%s\t%s", FlagNamePrefixer(name, placeholder), usageWithDefault)
}
func flagFromFileEnv(filePath, envName string) (val string, ok bool) {
for _, envVar := range strings.Split(envName, ",") {
envVar = strings.TrimSpace(envVar)
if envVal, ok := syscall.Getenv(envVar); ok {
return envVal, true
}
}
for _, fileVar := range strings.Split(filePath, ",") {
if data, err := ioutil.ReadFile(fileVar); err == nil {
return string(data), true
}
}
return "", false
}

View file

@ -13,6 +13,7 @@ type BoolFlag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Destination *bool
}
@ -60,6 +61,7 @@ type BoolTFlag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Destination *bool
}
@ -107,6 +109,7 @@ type DurationFlag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value time.Duration
Destination *time.Duration
@ -155,6 +158,7 @@ type Float64Flag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value float64
Destination *float64
@ -200,11 +204,12 @@ func lookupFloat64(name string, set *flag.FlagSet) float64 {
// GenericFlag is a flag with type Generic
type GenericFlag struct {
Name string
Usage string
EnvVar string
Hidden bool
Value Generic
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value Generic
}
// String returns a readable representation of this value
@ -250,6 +255,7 @@ type Int64Flag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value int64
Destination *int64
@ -298,6 +304,7 @@ type IntFlag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value int
Destination *int
@ -343,11 +350,12 @@ func lookupInt(name string, set *flag.FlagSet) int {
// IntSliceFlag is a flag with type *IntSlice
type IntSliceFlag struct {
Name string
Usage string
EnvVar string
Hidden bool
Value *IntSlice
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value *IntSlice
}
// String returns a readable representation of this value
@ -390,11 +398,12 @@ func lookupIntSlice(name string, set *flag.FlagSet) []int {
// Int64SliceFlag is a flag with type *Int64Slice
type Int64SliceFlag struct {
Name string
Usage string
EnvVar string
Hidden bool
Value *Int64Slice
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value *Int64Slice
}
// String returns a readable representation of this value
@ -440,6 +449,7 @@ type StringFlag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value string
Destination *string
@ -485,11 +495,12 @@ func lookupString(name string, set *flag.FlagSet) string {
// StringSliceFlag is a flag with type *StringSlice
type StringSliceFlag struct {
Name string
Usage string
EnvVar string
Hidden bool
Value *StringSlice
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value *StringSlice
}
// String returns a readable representation of this value
@ -535,6 +546,7 @@ type Uint64Flag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value uint64
Destination *uint64
@ -583,6 +595,7 @@ type UintFlag struct {
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
Value uint
Destination *uint

View file

@ -23,6 +23,22 @@ type CommandNotFoundFunc func(*Context, string)
// is displayed and the execution is interrupted.
type OnUsageErrorFunc func(context *Context, err error, isSubcommand bool) error
// ExitErrHandlerFunc is executed if provided in order to handle ExitError values
// returned by Actions and Before/After functions.
type ExitErrHandlerFunc func(context *Context, err error)
// FlagStringFunc is used by the help generation to display a flag, which is
// expected to be a single line.
type FlagStringFunc func(Flag) string
// FlagNamePrefixFunc is used by the default FlagStringFunc to create prefix
// text for a flag's full name.
type FlagNamePrefixFunc func(fullName, placeholder string) string
// FlagEnvHintFunc is used by the default FlagStringFunc to annotate flag help
// with the environment variable details.
type FlagEnvHintFunc func(envVar, str string) string
// FlagFileHintFunc is used by the default FlagStringFunc to annotate flag help
// with the file path details.
type FlagFileHintFunc func(filePath, str string) string

1
vendor/github.com/urfave/cli/generate-flag-types generated vendored Normal file → Executable file
View file

@ -142,6 +142,7 @@ def _write_cli_flag_types(outfile, types):
Name string
Usage string
EnvVar string
FilePath string
Hidden bool
""".format(**typedef))

0
vendor/github.com/urfave/cli/runtests generated vendored Normal file → Executable file
View file

29
vendor/github.com/urfave/cli/sort.go generated vendored Normal file
View file

@ -0,0 +1,29 @@
package cli
import "unicode"
// lexicographicLess compares strings alphabetically considering case.
func lexicographicLess(i, j string) bool {
iRunes := []rune(i)
jRunes := []rune(j)
lenShared := len(iRunes)
if lenShared > len(jRunes) {
lenShared = len(jRunes)
}
for index := 0; index < lenShared; index++ {
ir := iRunes[index]
jr := jRunes[index]
if lir, ljr := unicode.ToLower(ir), unicode.ToLower(jr); lir != ljr {
return lir < ljr
}
if ir != jr {
return ir < jr
}
}
return i < j
}

View file

@ -1,713 +0,0 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
package main
// This program generates table.go and table_test.go based on the authoritative
// public suffix list at https://publicsuffix.org/list/effective_tld_names.dat
//
// The version is derived from
// https://api.github.com/repos/publicsuffix/list/commits?path=public_suffix_list.dat
// and a human-readable form is at
// https://github.com/publicsuffix/list/commits/master/public_suffix_list.dat
//
// To fetch a particular git revision, such as 5c70ccd250, pass
// -url "https://raw.githubusercontent.com/publicsuffix/list/5c70ccd250/public_suffix_list.dat"
// and -version "an explicit version string".
import (
"bufio"
"bytes"
"flag"
"fmt"
"go/format"
"io"
"io/ioutil"
"net/http"
"os"
"regexp"
"sort"
"strings"
"golang.org/x/net/idna"
)
const (
// These sum of these four values must be no greater than 32.
nodesBitsChildren = 10
nodesBitsICANN = 1
nodesBitsTextOffset = 15
nodesBitsTextLength = 6
// These sum of these four values must be no greater than 32.
childrenBitsWildcard = 1
childrenBitsNodeType = 2
childrenBitsHi = 14
childrenBitsLo = 14
)
var (
maxChildren int
maxTextOffset int
maxTextLength int
maxHi uint32
maxLo uint32
)
func max(a, b int) int {
if a < b {
return b
}
return a
}
func u32max(a, b uint32) uint32 {
if a < b {
return b
}
return a
}
const (
nodeTypeNormal = 0
nodeTypeException = 1
nodeTypeParentOnly = 2
numNodeType = 3
)
func nodeTypeStr(n int) string {
switch n {
case nodeTypeNormal:
return "+"
case nodeTypeException:
return "!"
case nodeTypeParentOnly:
return "o"
}
panic("unreachable")
}
const (
defaultURL = "https://publicsuffix.org/list/effective_tld_names.dat"
gitCommitURL = "https://api.github.com/repos/publicsuffix/list/commits?path=public_suffix_list.dat"
)
var (
labelEncoding = map[string]uint32{}
labelsList = []string{}
labelsMap = map[string]bool{}
rules = []string{}
// validSuffixRE is used to check that the entries in the public suffix
// list are in canonical form (after Punycode encoding). Specifically,
// capital letters are not allowed.
validSuffixRE = regexp.MustCompile(`^[a-z0-9_\!\*\-\.]+$`)
shaRE = regexp.MustCompile(`"sha":"([^"]+)"`)
dateRE = regexp.MustCompile(`"committer":{[^{]+"date":"([^"]+)"`)
comments = flag.Bool("comments", false, "generate table.go comments, for debugging")
subset = flag.Bool("subset", false, "generate only a subset of the full table, for debugging")
url = flag.String("url", defaultURL, "URL of the publicsuffix.org list. If empty, stdin is read instead")
v = flag.Bool("v", false, "verbose output (to stderr)")
version = flag.String("version", "", "the effective_tld_names.dat version")
)
func main() {
if err := main1(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func main1() error {
flag.Parse()
if nodesBitsTextLength+nodesBitsTextOffset+nodesBitsICANN+nodesBitsChildren > 32 {
return fmt.Errorf("not enough bits to encode the nodes table")
}
if childrenBitsLo+childrenBitsHi+childrenBitsNodeType+childrenBitsWildcard > 32 {
return fmt.Errorf("not enough bits to encode the children table")
}
if *version == "" {
if *url != defaultURL {
return fmt.Errorf("-version was not specified, and the -url is not the default one")
}
sha, date, err := gitCommit()
if err != nil {
return err
}
*version = fmt.Sprintf("publicsuffix.org's public_suffix_list.dat, git revision %s (%s)", sha, date)
}
var r io.Reader = os.Stdin
if *url != "" {
res, err := http.Get(*url)
if err != nil {
return err
}
if res.StatusCode != http.StatusOK {
return fmt.Errorf("bad GET status for %s: %d", *url, res.Status)
}
r = res.Body
defer res.Body.Close()
}
var root node
icann := false
br := bufio.NewReader(r)
for {
s, err := br.ReadString('\n')
if err != nil {
if err == io.EOF {
break
}
return err
}
s = strings.TrimSpace(s)
if strings.Contains(s, "BEGIN ICANN DOMAINS") {
icann = true
continue
}
if strings.Contains(s, "END ICANN DOMAINS") {
icann = false
continue
}
if s == "" || strings.HasPrefix(s, "//") {
continue
}
s, err = idna.ToASCII(s)
if err != nil {
return err
}
if !validSuffixRE.MatchString(s) {
return fmt.Errorf("bad publicsuffix.org list data: %q", s)
}
if *subset {
switch {
case s == "ac.jp" || strings.HasSuffix(s, ".ac.jp"):
case s == "ak.us" || strings.HasSuffix(s, ".ak.us"):
case s == "ao" || strings.HasSuffix(s, ".ao"):
case s == "ar" || strings.HasSuffix(s, ".ar"):
case s == "arpa" || strings.HasSuffix(s, ".arpa"):
case s == "cy" || strings.HasSuffix(s, ".cy"):
case s == "dyndns.org" || strings.HasSuffix(s, ".dyndns.org"):
case s == "jp":
case s == "kobe.jp" || strings.HasSuffix(s, ".kobe.jp"):
case s == "kyoto.jp" || strings.HasSuffix(s, ".kyoto.jp"):
case s == "om" || strings.HasSuffix(s, ".om"):
case s == "uk" || strings.HasSuffix(s, ".uk"):
case s == "uk.com" || strings.HasSuffix(s, ".uk.com"):
case s == "tw" || strings.HasSuffix(s, ".tw"):
case s == "zw" || strings.HasSuffix(s, ".zw"):
case s == "xn--p1ai" || strings.HasSuffix(s, ".xn--p1ai"):
// xn--p1ai is Russian-Cyrillic "рф".
default:
continue
}
}
rules = append(rules, s)
nt, wildcard := nodeTypeNormal, false
switch {
case strings.HasPrefix(s, "*."):
s, nt = s[2:], nodeTypeParentOnly
wildcard = true
case strings.HasPrefix(s, "!"):
s, nt = s[1:], nodeTypeException
}
labels := strings.Split(s, ".")
for n, i := &root, len(labels)-1; i >= 0; i-- {
label := labels[i]
n = n.child(label)
if i == 0 {
if nt != nodeTypeParentOnly && n.nodeType == nodeTypeParentOnly {
n.nodeType = nt
}
n.icann = n.icann && icann
n.wildcard = n.wildcard || wildcard
}
labelsMap[label] = true
}
}
labelsList = make([]string, 0, len(labelsMap))
for label := range labelsMap {
labelsList = append(labelsList, label)
}
sort.Strings(labelsList)
if err := generate(printReal, &root, "table.go"); err != nil {
return err
}
if err := generate(printTest, &root, "table_test.go"); err != nil {
return err
}
return nil
}
func generate(p func(io.Writer, *node) error, root *node, filename string) error {
buf := new(bytes.Buffer)
if err := p(buf, root); err != nil {
return err
}
b, err := format.Source(buf.Bytes())
if err != nil {
return err
}
return ioutil.WriteFile(filename, b, 0644)
}
func gitCommit() (sha, date string, retErr error) {
res, err := http.Get(gitCommitURL)
if err != nil {
return "", "", err
}
if res.StatusCode != http.StatusOK {
return "", "", fmt.Errorf("bad GET status for %s: %d", gitCommitURL, res.Status)
}
defer res.Body.Close()
b, err := ioutil.ReadAll(res.Body)
if err != nil {
return "", "", err
}
if m := shaRE.FindSubmatch(b); m != nil {
sha = string(m[1])
}
if m := dateRE.FindSubmatch(b); m != nil {
date = string(m[1])
}
if sha == "" || date == "" {
retErr = fmt.Errorf("could not find commit SHA and date in %s", gitCommitURL)
}
return sha, date, retErr
}
func printTest(w io.Writer, n *node) error {
fmt.Fprintf(w, "// generated by go run gen.go; DO NOT EDIT\n\n")
fmt.Fprintf(w, "package publicsuffix\n\nvar rules = [...]string{\n")
for _, rule := range rules {
fmt.Fprintf(w, "%q,\n", rule)
}
fmt.Fprintf(w, "}\n\nvar nodeLabels = [...]string{\n")
if err := n.walk(w, printNodeLabel); err != nil {
return err
}
fmt.Fprintf(w, "}\n")
return nil
}
func printReal(w io.Writer, n *node) error {
const header = `// generated by go run gen.go; DO NOT EDIT
package publicsuffix
const version = %q
const (
nodesBitsChildren = %d
nodesBitsICANN = %d
nodesBitsTextOffset = %d
nodesBitsTextLength = %d
childrenBitsWildcard = %d
childrenBitsNodeType = %d
childrenBitsHi = %d
childrenBitsLo = %d
)
const (
nodeTypeNormal = %d
nodeTypeException = %d
nodeTypeParentOnly = %d
)
// numTLD is the number of top level domains.
const numTLD = %d
`
fmt.Fprintf(w, header, *version,
nodesBitsChildren, nodesBitsICANN, nodesBitsTextOffset, nodesBitsTextLength,
childrenBitsWildcard, childrenBitsNodeType, childrenBitsHi, childrenBitsLo,
nodeTypeNormal, nodeTypeException, nodeTypeParentOnly, len(n.children))
text := combineText(labelsList)
if text == "" {
return fmt.Errorf("internal error: makeText returned no text")
}
for _, label := range labelsList {
offset, length := strings.Index(text, label), len(label)
if offset < 0 {
return fmt.Errorf("internal error: could not find %q in text %q", label, text)
}
maxTextOffset, maxTextLength = max(maxTextOffset, offset), max(maxTextLength, length)
if offset >= 1<<nodesBitsTextOffset {
return fmt.Errorf("text offset %d is too large, or nodeBitsTextOffset is too small", offset)
}
if length >= 1<<nodesBitsTextLength {
return fmt.Errorf("text length %d is too large, or nodeBitsTextLength is too small", length)
}
labelEncoding[label] = uint32(offset)<<nodesBitsTextLength | uint32(length)
}
fmt.Fprintf(w, "// Text is the combined text of all labels.\nconst text = ")
for len(text) > 0 {
n, plus := len(text), ""
if n > 64 {
n, plus = 64, " +"
}
fmt.Fprintf(w, "%q%s\n", text[:n], plus)
text = text[n:]
}
if err := n.walk(w, assignIndexes); err != nil {
return err
}
fmt.Fprintf(w, `
// nodes is the list of nodes. Each node is represented as a uint32, which
// encodes the node's children, wildcard bit and node type (as an index into
// the children array), ICANN bit and text.
//
// If the table was generated with the -comments flag, there is a //-comment
// after each node's data. In it is the nodes-array indexes of the children,
// formatted as (n0x1234-n0x1256), with * denoting the wildcard bit. The
// nodeType is printed as + for normal, ! for exception, and o for parent-only
// nodes that have children but don't match a domain label in their own right.
// An I denotes an ICANN domain.
//
// The layout within the uint32, from MSB to LSB, is:
// [%2d bits] unused
// [%2d bits] children index
// [%2d bits] ICANN bit
// [%2d bits] text index
// [%2d bits] text length
var nodes = [...]uint32{
`,
32-nodesBitsChildren-nodesBitsICANN-nodesBitsTextOffset-nodesBitsTextLength,
nodesBitsChildren, nodesBitsICANN, nodesBitsTextOffset, nodesBitsTextLength)
if err := n.walk(w, printNode); err != nil {
return err
}
fmt.Fprintf(w, `}
// children is the list of nodes' children, the parent's wildcard bit and the
// parent's node type. If a node has no children then their children index
// will be in the range [0, 6), depending on the wildcard bit and node type.
//
// The layout within the uint32, from MSB to LSB, is:
// [%2d bits] unused
// [%2d bits] wildcard bit
// [%2d bits] node type
// [%2d bits] high nodes index (exclusive) of children
// [%2d bits] low nodes index (inclusive) of children
var children=[...]uint32{
`,
32-childrenBitsWildcard-childrenBitsNodeType-childrenBitsHi-childrenBitsLo,
childrenBitsWildcard, childrenBitsNodeType, childrenBitsHi, childrenBitsLo)
for i, c := range childrenEncoding {
s := "---------------"
lo := c & (1<<childrenBitsLo - 1)
hi := (c >> childrenBitsLo) & (1<<childrenBitsHi - 1)
if lo != hi {
s = fmt.Sprintf("n0x%04x-n0x%04x", lo, hi)
}
nodeType := int(c>>(childrenBitsLo+childrenBitsHi)) & (1<<childrenBitsNodeType - 1)
wildcard := c>>(childrenBitsLo+childrenBitsHi+childrenBitsNodeType) != 0
if *comments {
fmt.Fprintf(w, "0x%08x, // c0x%04x (%s)%s %s\n",
c, i, s, wildcardStr(wildcard), nodeTypeStr(nodeType))
} else {
fmt.Fprintf(w, "0x%x,\n", c)
}
}
fmt.Fprintf(w, "}\n\n")
fmt.Fprintf(w, "// max children %d (capacity %d)\n", maxChildren, 1<<nodesBitsChildren-1)
fmt.Fprintf(w, "// max text offset %d (capacity %d)\n", maxTextOffset, 1<<nodesBitsTextOffset-1)
fmt.Fprintf(w, "// max text length %d (capacity %d)\n", maxTextLength, 1<<nodesBitsTextLength-1)
fmt.Fprintf(w, "// max hi %d (capacity %d)\n", maxHi, 1<<childrenBitsHi-1)
fmt.Fprintf(w, "// max lo %d (capacity %d)\n", maxLo, 1<<childrenBitsLo-1)
return nil
}
type node struct {
label string
nodeType int
icann bool
wildcard bool
// nodesIndex and childrenIndex are the index of this node in the nodes
// and the index of its children offset/length in the children arrays.
nodesIndex, childrenIndex int
// firstChild is the index of this node's first child, or zero if this
// node has no children.
firstChild int
// children are the node's children, in strictly increasing node label order.
children []*node
}
func (n *node) walk(w io.Writer, f func(w1 io.Writer, n1 *node) error) error {
if err := f(w, n); err != nil {
return err
}
for _, c := range n.children {
if err := c.walk(w, f); err != nil {
return err
}
}
return nil
}
// child returns the child of n with the given label. The child is created if
// it did not exist beforehand.
func (n *node) child(label string) *node {
for _, c := range n.children {
if c.label == label {
return c
}
}
c := &node{
label: label,
nodeType: nodeTypeParentOnly,
icann: true,
}
n.children = append(n.children, c)
sort.Sort(byLabel(n.children))
return c
}
type byLabel []*node
func (b byLabel) Len() int { return len(b) }
func (b byLabel) Swap(i, j int) { b[i], b[j] = b[j], b[i] }
func (b byLabel) Less(i, j int) bool { return b[i].label < b[j].label }
var nextNodesIndex int
// childrenEncoding are the encoded entries in the generated children array.
// All these pre-defined entries have no children.
var childrenEncoding = []uint32{
0 << (childrenBitsLo + childrenBitsHi), // Without wildcard bit, nodeTypeNormal.
1 << (childrenBitsLo + childrenBitsHi), // Without wildcard bit, nodeTypeException.
2 << (childrenBitsLo + childrenBitsHi), // Without wildcard bit, nodeTypeParentOnly.
4 << (childrenBitsLo + childrenBitsHi), // With wildcard bit, nodeTypeNormal.
5 << (childrenBitsLo + childrenBitsHi), // With wildcard bit, nodeTypeException.
6 << (childrenBitsLo + childrenBitsHi), // With wildcard bit, nodeTypeParentOnly.
}
var firstCallToAssignIndexes = true
func assignIndexes(w io.Writer, n *node) error {
if len(n.children) != 0 {
// Assign nodesIndex.
n.firstChild = nextNodesIndex
for _, c := range n.children {
c.nodesIndex = nextNodesIndex
nextNodesIndex++
}
// The root node's children is implicit.
if firstCallToAssignIndexes {
firstCallToAssignIndexes = false
return nil
}
// Assign childrenIndex.
maxChildren = max(maxChildren, len(childrenEncoding))
if len(childrenEncoding) >= 1<<nodesBitsChildren {
return fmt.Errorf("children table size %d is too large, or nodeBitsChildren is too small", len(childrenEncoding))
}
n.childrenIndex = len(childrenEncoding)
lo := uint32(n.firstChild)
hi := lo + uint32(len(n.children))
maxLo, maxHi = u32max(maxLo, lo), u32max(maxHi, hi)
if lo >= 1<<childrenBitsLo {
return fmt.Errorf("children lo %d is too large, or childrenBitsLo is too small", lo)
}
if hi >= 1<<childrenBitsHi {
return fmt.Errorf("children hi %d is too large, or childrenBitsHi is too small", hi)
}
enc := hi<<childrenBitsLo | lo
enc |= uint32(n.nodeType) << (childrenBitsLo + childrenBitsHi)
if n.wildcard {
enc |= 1 << (childrenBitsLo + childrenBitsHi + childrenBitsNodeType)
}
childrenEncoding = append(childrenEncoding, enc)
} else {
n.childrenIndex = n.nodeType
if n.wildcard {
n.childrenIndex += numNodeType
}
}
return nil
}
func printNode(w io.Writer, n *node) error {
for _, c := range n.children {
s := "---------------"
if len(c.children) != 0 {
s = fmt.Sprintf("n0x%04x-n0x%04x", c.firstChild, c.firstChild+len(c.children))
}
encoding := labelEncoding[c.label]
if c.icann {
encoding |= 1 << (nodesBitsTextLength + nodesBitsTextOffset)
}
encoding |= uint32(c.childrenIndex) << (nodesBitsTextLength + nodesBitsTextOffset + nodesBitsICANN)
if *comments {
fmt.Fprintf(w, "0x%08x, // n0x%04x c0x%04x (%s)%s %s %s %s\n",
encoding, c.nodesIndex, c.childrenIndex, s, wildcardStr(c.wildcard),
nodeTypeStr(c.nodeType), icannStr(c.icann), c.label,
)
} else {
fmt.Fprintf(w, "0x%x,\n", encoding)
}
}
return nil
}
func printNodeLabel(w io.Writer, n *node) error {
for _, c := range n.children {
fmt.Fprintf(w, "%q,\n", c.label)
}
return nil
}
func icannStr(icann bool) string {
if icann {
return "I"
}
return " "
}
func wildcardStr(wildcard bool) string {
if wildcard {
return "*"
}
return " "
}
// combineText combines all the strings in labelsList to form one giant string.
// Overlapping strings will be merged: "arpa" and "parliament" could yield
// "arparliament".
func combineText(labelsList []string) string {
beforeLength := 0
for _, s := range labelsList {
beforeLength += len(s)
}
text := crush(removeSubstrings(labelsList))
if *v {
fmt.Fprintf(os.Stderr, "crushed %d bytes to become %d bytes\n", beforeLength, len(text))
}
return text
}
type byLength []string
func (s byLength) Len() int { return len(s) }
func (s byLength) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
func (s byLength) Less(i, j int) bool { return len(s[i]) < len(s[j]) }
// removeSubstrings returns a copy of its input with any strings removed
// that are substrings of other provided strings.
func removeSubstrings(input []string) []string {
// Make a copy of input.
ss := append(make([]string, 0, len(input)), input...)
sort.Sort(byLength(ss))
for i, shortString := range ss {
// For each string, only consider strings higher than it in sort order, i.e.
// of equal length or greater.
for _, longString := range ss[i+1:] {
if strings.Contains(longString, shortString) {
ss[i] = ""
break
}
}
}
// Remove the empty strings.
sort.Strings(ss)
for len(ss) > 0 && ss[0] == "" {
ss = ss[1:]
}
return ss
}
// crush combines a list of strings, taking advantage of overlaps. It returns a
// single string that contains each input string as a substring.
func crush(ss []string) string {
maxLabelLen := 0
for _, s := range ss {
if maxLabelLen < len(s) {
maxLabelLen = len(s)
}
}
for prefixLen := maxLabelLen; prefixLen > 0; prefixLen-- {
prefixes := makePrefixMap(ss, prefixLen)
for i, s := range ss {
if len(s) <= prefixLen {
continue
}
mergeLabel(ss, i, prefixLen, prefixes)
}
}
return strings.Join(ss, "")
}
// mergeLabel merges the label at ss[i] with the first available matching label
// in prefixMap, where the last "prefixLen" characters in ss[i] match the first
// "prefixLen" characters in the matching label.
// It will merge ss[i] repeatedly until no more matches are available.
// All matching labels merged into ss[i] are replaced by "".
func mergeLabel(ss []string, i, prefixLen int, prefixes prefixMap) {
s := ss[i]
suffix := s[len(s)-prefixLen:]
for _, j := range prefixes[suffix] {
// Empty strings mean "already used." Also avoid merging with self.
if ss[j] == "" || i == j {
continue
}
if *v {
fmt.Fprintf(os.Stderr, "%d-length overlap at (%4d,%4d): %q and %q share %q\n",
prefixLen, i, j, ss[i], ss[j], suffix)
}
ss[i] += ss[j][prefixLen:]
ss[j] = ""
// ss[i] has a new suffix, so merge again if possible.
// Note: we only have to merge again at the same prefix length. Shorter
// prefix lengths will be handled in the next iteration of crush's for loop.
// Can there be matches for longer prefix lengths, introduced by the merge?
// I believe that any such matches would by necessity have been eliminated
// during substring removal or merged at a higher prefix length. For
// instance, in crush("abc", "cde", "bcdef"), combining "abc" and "cde"
// would yield "abcde", which could be merged with "bcdef." However, in
// practice "cde" would already have been elimintated by removeSubstrings.
mergeLabel(ss, i, prefixLen, prefixes)
return
}
}
// prefixMap maps from a prefix to a list of strings containing that prefix. The
// list of strings is represented as indexes into a slice of strings stored
// elsewhere.
type prefixMap map[string][]int
// makePrefixMap constructs a prefixMap from a slice of strings.
func makePrefixMap(ss []string, prefixLen int) prefixMap {
prefixes := make(prefixMap)
for i, s := range ss {
// We use < rather than <= because if a label matches on a prefix equal to
// its full length, that's actually a substring match handled by
// removeSubstrings.
if prefixLen < len(s) {
prefix := s[:prefixLen]
prefixes[prefix] = append(prefixes[prefix], i)
}
}
return prefixes
}

File diff suppressed because it is too large Load diff

View file

@ -33,32 +33,32 @@ var (
ISO8859_8I encoding.Encoding = &iso8859_8I
iso8859_6E = internal.Encoding{
Encoding: ISO8859_6,
Name: "ISO-8859-6E",
MIB: identifier.ISO88596E,
ISO8859_6,
"ISO-8859-6E",
identifier.ISO88596E,
}
iso8859_6I = internal.Encoding{
Encoding: ISO8859_6,
Name: "ISO-8859-6I",
MIB: identifier.ISO88596I,
ISO8859_6,
"ISO-8859-6I",
identifier.ISO88596I,
}
iso8859_8E = internal.Encoding{
Encoding: ISO8859_8,
Name: "ISO-8859-8E",
MIB: identifier.ISO88598E,
ISO8859_8,
"ISO-8859-8E",
identifier.ISO88598E,
}
iso8859_8I = internal.Encoding{
Encoding: ISO8859_8,
Name: "ISO-8859-8I",
MIB: identifier.ISO88598I,
ISO8859_8,
"ISO-8859-8I",
identifier.ISO88598I,
}
)
// All is a list of all defined encodings in this package.
var All []encoding.Encoding = listAll
var All = listAll
// TODO: implement these encodings, in order of importance.
// ASCII, ISO8859_1: Rather common. Close to Windows 1252.
@ -70,8 +70,8 @@ type utf8Enc struct {
data [3]byte
}
// Charmap is an 8-bit character set encoding.
type Charmap struct {
// charmap describes an 8-bit character set encoding.
type charmap struct {
// name is the encoding's name.
name string
// mib is the encoding type of this encoder.
@ -79,7 +79,7 @@ type Charmap struct {
// asciiSuperset states whether the encoding is a superset of ASCII.
asciiSuperset bool
// low is the lower bound of the encoded byte for a non-ASCII rune. If
// Charmap.asciiSuperset is true then this will be 0x80, otherwise 0x00.
// charmap.asciiSuperset is true then this will be 0x80, otherwise 0x00.
low uint8
// replacement is the encoded replacement character.
replacement byte
@ -91,30 +91,26 @@ type Charmap struct {
encode [256]uint32
}
// NewDecoder implements the encoding.Encoding interface.
func (m *Charmap) NewDecoder() *encoding.Decoder {
func (m *charmap) NewDecoder() *encoding.Decoder {
return &encoding.Decoder{Transformer: charmapDecoder{charmap: m}}
}
// NewEncoder implements the encoding.Encoding interface.
func (m *Charmap) NewEncoder() *encoding.Encoder {
func (m *charmap) NewEncoder() *encoding.Encoder {
return &encoding.Encoder{Transformer: charmapEncoder{charmap: m}}
}
// String returns the Charmap's name.
func (m *Charmap) String() string {
func (m *charmap) String() string {
return m.name
}
// ID implements an internal interface.
func (m *Charmap) ID() (mib identifier.MIB, other string) {
func (m *charmap) ID() (mib identifier.MIB, other string) {
return m.mib, ""
}
// charmapDecoder implements transform.Transformer by decoding to UTF-8.
type charmapDecoder struct {
transform.NopResetter
charmap *Charmap
charmap *charmap
}
func (m charmapDecoder) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
@ -146,22 +142,10 @@ func (m charmapDecoder) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int,
return nDst, nSrc, err
}
// DecodeByte returns the Charmap's rune decoding of the byte b.
func (m *Charmap) DecodeByte(b byte) rune {
switch x := &m.decode[b]; x.len {
case 1:
return rune(x.data[0])
case 2:
return rune(x.data[0]&0x1f)<<6 | rune(x.data[1]&0x3f)
default:
return rune(x.data[0]&0x0f)<<12 | rune(x.data[1]&0x3f)<<6 | rune(x.data[2]&0x3f)
}
}
// charmapEncoder implements transform.Transformer by encoding from UTF-8.
type charmapEncoder struct {
transform.NopResetter
charmap *Charmap
charmap *charmap
}
func (m charmapEncoder) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
@ -223,27 +207,3 @@ loop:
}
return nDst, nSrc, err
}
// EncodeRune returns the Charmap's byte encoding of the rune r. ok is whether
// r is in the Charmap's repertoire. If not, b is set to the Charmap's
// replacement byte. This is often the ASCII substitute character '\x1a'.
func (m *Charmap) EncodeRune(r rune) (b byte, ok bool) {
if r < utf8.RuneSelf && m.asciiSuperset {
return byte(r), true
}
for low, high := int(m.low), 0x100; ; {
if low >= high {
return m.replacement, false
}
mid := (low + high) / 2
got := m.encode[mid]
gotRune := rune(got & (1<<24 - 1))
if gotRune < r {
low = mid + 1
} else if gotRune > r {
high = mid
} else {
return byte(got >> 24), true
}
}
}

View file

@ -1,556 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
package main
import (
"bufio"
"fmt"
"log"
"net/http"
"sort"
"strings"
"unicode/utf8"
"golang.org/x/text/encoding"
"golang.org/x/text/internal/gen"
)
const ascii = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" +
"\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" +
` !"#$%&'()*+,-./0123456789:;<=>?` +
`@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_` +
"`abcdefghijklmnopqrstuvwxyz{|}~\u007f"
var encodings = []struct {
name string
mib string
comment string
varName string
replacement byte
mapping string
}{
{
"IBM Code Page 037",
"IBM037",
"",
"CodePage037",
0x3f,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM037-2.1.2.ucm",
},
{
"IBM Code Page 437",
"PC8CodePage437",
"",
"CodePage437",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM437-2.1.2.ucm",
},
{
"IBM Code Page 850",
"PC850Multilingual",
"",
"CodePage850",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM850-2.1.2.ucm",
},
{
"IBM Code Page 852",
"PCp852",
"",
"CodePage852",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM852-2.1.2.ucm",
},
{
"IBM Code Page 855",
"IBM855",
"",
"CodePage855",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM855-2.1.2.ucm",
},
{
"Windows Code Page 858", // PC latin1 with Euro
"IBM00858",
"",
"CodePage858",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/windows-858-2000.ucm",
},
{
"IBM Code Page 860",
"IBM860",
"",
"CodePage860",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM860-2.1.2.ucm",
},
{
"IBM Code Page 862",
"PC862LatinHebrew",
"",
"CodePage862",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM862-2.1.2.ucm",
},
{
"IBM Code Page 863",
"IBM863",
"",
"CodePage863",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM863-2.1.2.ucm",
},
{
"IBM Code Page 865",
"IBM865",
"",
"CodePage865",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM865-2.1.2.ucm",
},
{
"IBM Code Page 866",
"IBM866",
"",
"CodePage866",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-ibm866.txt",
},
{
"IBM Code Page 1047",
"IBM1047",
"",
"CodePage1047",
0x3f,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/glibc-IBM1047-2.1.2.ucm",
},
{
"IBM Code Page 1140",
"IBM01140",
"",
"CodePage1140",
0x3f,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/ibm-1140_P100-1997.ucm",
},
{
"ISO 8859-1",
"ISOLatin1",
"",
"ISO8859_1",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/iso-8859_1-1998.ucm",
},
{
"ISO 8859-2",
"ISOLatin2",
"",
"ISO8859_2",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-2.txt",
},
{
"ISO 8859-3",
"ISOLatin3",
"",
"ISO8859_3",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-3.txt",
},
{
"ISO 8859-4",
"ISOLatin4",
"",
"ISO8859_4",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-4.txt",
},
{
"ISO 8859-5",
"ISOLatinCyrillic",
"",
"ISO8859_5",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-5.txt",
},
{
"ISO 8859-6",
"ISOLatinArabic",
"",
"ISO8859_6,ISO8859_6E,ISO8859_6I",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-6.txt",
},
{
"ISO 8859-7",
"ISOLatinGreek",
"",
"ISO8859_7",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-7.txt",
},
{
"ISO 8859-8",
"ISOLatinHebrew",
"",
"ISO8859_8,ISO8859_8E,ISO8859_8I",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-8.txt",
},
{
"ISO 8859-9",
"ISOLatin5",
"",
"ISO8859_9",
encoding.ASCIISub,
"http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/iso-8859_9-1999.ucm",
},
{
"ISO 8859-10",
"ISOLatin6",
"",
"ISO8859_10",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-10.txt",
},
{
"ISO 8859-13",
"ISO885913",
"",
"ISO8859_13",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-13.txt",
},
{
"ISO 8859-14",
"ISO885914",
"",
"ISO8859_14",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-14.txt",
},
{
"ISO 8859-15",
"ISO885915",
"",
"ISO8859_15",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-15.txt",
},
{
"ISO 8859-16",
"ISO885916",
"",
"ISO8859_16",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-iso-8859-16.txt",
},
{
"KOI8-R",
"KOI8R",
"",
"KOI8R",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-koi8-r.txt",
},
{
"KOI8-U",
"KOI8U",
"",
"KOI8U",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-koi8-u.txt",
},
{
"Macintosh",
"Macintosh",
"",
"Macintosh",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-macintosh.txt",
},
{
"Macintosh Cyrillic",
"MacintoshCyrillic",
"",
"MacintoshCyrillic",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-x-mac-cyrillic.txt",
},
{
"Windows 874",
"Windows874",
"",
"Windows874",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-874.txt",
},
{
"Windows 1250",
"Windows1250",
"",
"Windows1250",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1250.txt",
},
{
"Windows 1251",
"Windows1251",
"",
"Windows1251",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1251.txt",
},
{
"Windows 1252",
"Windows1252",
"",
"Windows1252",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1252.txt",
},
{
"Windows 1253",
"Windows1253",
"",
"Windows1253",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1253.txt",
},
{
"Windows 1254",
"Windows1254",
"",
"Windows1254",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1254.txt",
},
{
"Windows 1255",
"Windows1255",
"",
"Windows1255",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1255.txt",
},
{
"Windows 1256",
"Windows1256",
"",
"Windows1256",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1256.txt",
},
{
"Windows 1257",
"Windows1257",
"",
"Windows1257",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1257.txt",
},
{
"Windows 1258",
"Windows1258",
"",
"Windows1258",
encoding.ASCIISub,
"http://encoding.spec.whatwg.org/index-windows-1258.txt",
},
{
"X-User-Defined",
"XUserDefined",
"It is defined at http://encoding.spec.whatwg.org/#x-user-defined",
"XUserDefined",
encoding.ASCIISub,
ascii +
"\uf780\uf781\uf782\uf783\uf784\uf785\uf786\uf787" +
"\uf788\uf789\uf78a\uf78b\uf78c\uf78d\uf78e\uf78f" +
"\uf790\uf791\uf792\uf793\uf794\uf795\uf796\uf797" +
"\uf798\uf799\uf79a\uf79b\uf79c\uf79d\uf79e\uf79f" +
"\uf7a0\uf7a1\uf7a2\uf7a3\uf7a4\uf7a5\uf7a6\uf7a7" +
"\uf7a8\uf7a9\uf7aa\uf7ab\uf7ac\uf7ad\uf7ae\uf7af" +
"\uf7b0\uf7b1\uf7b2\uf7b3\uf7b4\uf7b5\uf7b6\uf7b7" +
"\uf7b8\uf7b9\uf7ba\uf7bb\uf7bc\uf7bd\uf7be\uf7bf" +
"\uf7c0\uf7c1\uf7c2\uf7c3\uf7c4\uf7c5\uf7c6\uf7c7" +
"\uf7c8\uf7c9\uf7ca\uf7cb\uf7cc\uf7cd\uf7ce\uf7cf" +
"\uf7d0\uf7d1\uf7d2\uf7d3\uf7d4\uf7d5\uf7d6\uf7d7" +
"\uf7d8\uf7d9\uf7da\uf7db\uf7dc\uf7dd\uf7de\uf7df" +
"\uf7e0\uf7e1\uf7e2\uf7e3\uf7e4\uf7e5\uf7e6\uf7e7" +
"\uf7e8\uf7e9\uf7ea\uf7eb\uf7ec\uf7ed\uf7ee\uf7ef" +
"\uf7f0\uf7f1\uf7f2\uf7f3\uf7f4\uf7f5\uf7f6\uf7f7" +
"\uf7f8\uf7f9\uf7fa\uf7fb\uf7fc\uf7fd\uf7fe\uf7ff",
},
}
func getWHATWG(url string) string {
res, err := http.Get(url)
if err != nil {
log.Fatalf("%q: Get: %v", url, err)
}
defer res.Body.Close()
mapping := make([]rune, 128)
for i := range mapping {
mapping[i] = '\ufffd'
}
scanner := bufio.NewScanner(res.Body)
for scanner.Scan() {
s := strings.TrimSpace(scanner.Text())
if s == "" || s[0] == '#' {
continue
}
x, y := 0, 0
if _, err := fmt.Sscanf(s, "%d\t0x%x", &x, &y); err != nil {
log.Fatalf("could not parse %q", s)
}
if x < 0 || 128 <= x {
log.Fatalf("code %d is out of range", x)
}
if 0x80 <= y && y < 0xa0 {
// We diverge from the WHATWG spec by mapping control characters
// in the range [0x80, 0xa0) to U+FFFD.
continue
}
mapping[x] = rune(y)
}
return ascii + string(mapping)
}
func getUCM(url string) string {
res, err := http.Get(url)
if err != nil {
log.Fatalf("%q: Get: %v", url, err)
}
defer res.Body.Close()
mapping := make([]rune, 256)
for i := range mapping {
mapping[i] = '\ufffd'
}
charsFound := 0
scanner := bufio.NewScanner(res.Body)
for scanner.Scan() {
s := strings.TrimSpace(scanner.Text())
if s == "" || s[0] == '#' {
continue
}
var c byte
var r rune
if _, err := fmt.Sscanf(s, `<U%x> \x%x |0`, &r, &c); err != nil {
continue
}
mapping[c] = r
charsFound++
}
if charsFound < 200 {
log.Fatalf("%q: only %d characters found (wrong page format?)", url, charsFound)
}
return string(mapping)
}
func main() {
mibs := map[string]bool{}
all := []string{}
w := gen.NewCodeWriter()
defer w.WriteGoFile("tables.go", "charmap")
printf := func(s string, a ...interface{}) { fmt.Fprintf(w, s, a...) }
printf("import (\n")
printf("\t\"golang.org/x/text/encoding\"\n")
printf("\t\"golang.org/x/text/encoding/internal/identifier\"\n")
printf(")\n\n")
for _, e := range encodings {
varNames := strings.Split(e.varName, ",")
all = append(all, varNames...)
varName := varNames[0]
switch {
case strings.HasPrefix(e.mapping, "http://encoding.spec.whatwg.org/"):
e.mapping = getWHATWG(e.mapping)
case strings.HasPrefix(e.mapping, "http://source.icu-project.org/repos/icu/data/trunk/charset/data/ucm/"):
e.mapping = getUCM(e.mapping)
}
asciiSuperset, low := strings.HasPrefix(e.mapping, ascii), 0x00
if asciiSuperset {
low = 0x80
}
lvn := 1
if strings.HasPrefix(varName, "ISO") || strings.HasPrefix(varName, "KOI") {
lvn = 3
}
lowerVarName := strings.ToLower(varName[:lvn]) + varName[lvn:]
printf("// %s is the %s encoding.\n", varName, e.name)
if e.comment != "" {
printf("//\n// %s\n", e.comment)
}
printf("var %s *Charmap = &%s\n\nvar %s = Charmap{\nname: %q,\n",
varName, lowerVarName, lowerVarName, e.name)
if mibs[e.mib] {
log.Fatalf("MIB type %q declared multiple times.", e.mib)
}
printf("mib: identifier.%s,\n", e.mib)
printf("asciiSuperset: %t,\n", asciiSuperset)
printf("low: 0x%02x,\n", low)
printf("replacement: 0x%02x,\n", e.replacement)
printf("decode: [256]utf8Enc{\n")
i, backMapping := 0, map[rune]byte{}
for _, c := range e.mapping {
if _, ok := backMapping[c]; !ok && c != utf8.RuneError {
backMapping[c] = byte(i)
}
var buf [8]byte
n := utf8.EncodeRune(buf[:], c)
if n > 3 {
panic(fmt.Sprintf("rune %q (%U) is too long", c, c))
}
printf("{%d,[3]byte{0x%02x,0x%02x,0x%02x}},", n, buf[0], buf[1], buf[2])
if i%2 == 1 {
printf("\n")
}
i++
}
printf("},\n")
printf("encode: [256]uint32{\n")
encode := make([]uint32, 0, 256)
for c, i := range backMapping {
encode = append(encode, uint32(i)<<24|uint32(c))
}
sort.Sort(byRune(encode))
for len(encode) < cap(encode) {
encode = append(encode, encode[len(encode)-1])
}
for i, enc := range encode {
printf("0x%08x,", enc)
if i%8 == 7 {
printf("\n")
}
}
printf("},\n}\n")
// Add an estimate of the size of a single Charmap{} struct value, which
// includes two 256 elem arrays of 4 bytes and some extra fields, which
// align to 3 uint64s on 64-bit architectures.
w.Size += 2*4*256 + 3*8
}
// TODO: add proper line breaking.
printf("var listAll = []encoding.Encoding{\n%s,\n}\n\n", strings.Join(all, ",\n"))
}
type byRune []uint32
func (b byRune) Len() int { return len(b) }
func (b byRune) Less(i, j int) bool { return b[i]&0xffffff < b[j]&0xffffff }
func (b byRune) Swap(i, j int) { b[i], b[j] = b[j], b[i] }

View file

@ -1,4 +1,4 @@
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.
// This file was generated by go generate; DO NOT EDIT
package charmap
@ -7,185 +7,10 @@ import (
"golang.org/x/text/encoding/internal/identifier"
)
// CodePage037 is the IBM Code Page 037 encoding.
var CodePage037 *Charmap = &codePage037
var codePage037 = Charmap{
name: "IBM Code Page 037",
mib: identifier.IBM037,
asciiSuperset: false,
low: 0x00,
replacement: 0x3f,
decode: [256]utf8Enc{
{1, [3]byte{0x00, 0x00, 0x00}}, {1, [3]byte{0x01, 0x00, 0x00}},
{1, [3]byte{0x02, 0x00, 0x00}}, {1, [3]byte{0x03, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x9c, 0x00}}, {1, [3]byte{0x09, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x86, 0x00}}, {1, [3]byte{0x7f, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x97, 0x00}}, {2, [3]byte{0xc2, 0x8d, 0x00}},
{2, [3]byte{0xc2, 0x8e, 0x00}}, {1, [3]byte{0x0b, 0x00, 0x00}},
{1, [3]byte{0x0c, 0x00, 0x00}}, {1, [3]byte{0x0d, 0x00, 0x00}},
{1, [3]byte{0x0e, 0x00, 0x00}}, {1, [3]byte{0x0f, 0x00, 0x00}},
{1, [3]byte{0x10, 0x00, 0x00}}, {1, [3]byte{0x11, 0x00, 0x00}},
{1, [3]byte{0x12, 0x00, 0x00}}, {1, [3]byte{0x13, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x9d, 0x00}}, {2, [3]byte{0xc2, 0x85, 0x00}},
{1, [3]byte{0x08, 0x00, 0x00}}, {2, [3]byte{0xc2, 0x87, 0x00}},
{1, [3]byte{0x18, 0x00, 0x00}}, {1, [3]byte{0x19, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x92, 0x00}}, {2, [3]byte{0xc2, 0x8f, 0x00}},
{1, [3]byte{0x1c, 0x00, 0x00}}, {1, [3]byte{0x1d, 0x00, 0x00}},
{1, [3]byte{0x1e, 0x00, 0x00}}, {1, [3]byte{0x1f, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x80, 0x00}}, {2, [3]byte{0xc2, 0x81, 0x00}},
{2, [3]byte{0xc2, 0x82, 0x00}}, {2, [3]byte{0xc2, 0x83, 0x00}},
{2, [3]byte{0xc2, 0x84, 0x00}}, {1, [3]byte{0x0a, 0x00, 0x00}},
{1, [3]byte{0x17, 0x00, 0x00}}, {1, [3]byte{0x1b, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x88, 0x00}}, {2, [3]byte{0xc2, 0x89, 0x00}},
{2, [3]byte{0xc2, 0x8a, 0x00}}, {2, [3]byte{0xc2, 0x8b, 0x00}},
{2, [3]byte{0xc2, 0x8c, 0x00}}, {1, [3]byte{0x05, 0x00, 0x00}},
{1, [3]byte{0x06, 0x00, 0x00}}, {1, [3]byte{0x07, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x90, 0x00}}, {2, [3]byte{0xc2, 0x91, 0x00}},
{1, [3]byte{0x16, 0x00, 0x00}}, {2, [3]byte{0xc2, 0x93, 0x00}},
{2, [3]byte{0xc2, 0x94, 0x00}}, {2, [3]byte{0xc2, 0x95, 0x00}},
{2, [3]byte{0xc2, 0x96, 0x00}}, {1, [3]byte{0x04, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x98, 0x00}}, {2, [3]byte{0xc2, 0x99, 0x00}},
{2, [3]byte{0xc2, 0x9a, 0x00}}, {2, [3]byte{0xc2, 0x9b, 0x00}},
{1, [3]byte{0x14, 0x00, 0x00}}, {1, [3]byte{0x15, 0x00, 0x00}},
{2, [3]byte{0xc2, 0x9e, 0x00}}, {1, [3]byte{0x1a, 0x00, 0x00}},
{1, [3]byte{0x20, 0x00, 0x00}}, {2, [3]byte{0xc2, 0xa0, 0x00}},
{2, [3]byte{0xc3, 0xa2, 0x00}}, {2, [3]byte{0xc3, 0xa4, 0x00}},
{2, [3]byte{0xc3, 0xa0, 0x00}}, {2, [3]byte{0xc3, 0xa1, 0x00}},
{2, [3]byte{0xc3, 0xa3, 0x00}}, {2, [3]byte{0xc3, 0xa5, 0x00}},
{2, [3]byte{0xc3, 0xa7, 0x00}}, {2, [3]byte{0xc3, 0xb1, 0x00}},
{2, [3]byte{0xc2, 0xa2, 0x00}}, {1, [3]byte{0x2e, 0x00, 0x00}},
{1, [3]byte{0x3c, 0x00, 0x00}}, {1, [3]byte{0x28, 0x00, 0x00}},
{1, [3]byte{0x2b, 0x00, 0x00}}, {1, [3]byte{0x7c, 0x00, 0x00}},
{1, [3]byte{0x26, 0x00, 0x00}}, {2, [3]byte{0xc3, 0xa9, 0x00}},
{2, [3]byte{0xc3, 0xaa, 0x00}}, {2, [3]byte{0xc3, 0xab, 0x00}},
{2, [3]byte{0xc3, 0xa8, 0x00}}, {2, [3]byte{0xc3, 0xad, 0x00}},
{2, [3]byte{0xc3, 0xae, 0x00}}, {2, [3]byte{0xc3, 0xaf, 0x00}},
{2, [3]byte{0xc3, 0xac, 0x00}}, {2, [3]byte{0xc3, 0x9f, 0x00}},
{1, [3]byte{0x21, 0x00, 0x00}}, {1, [3]byte{0x24, 0x00, 0x00}},
{1, [3]byte{0x2a, 0x00, 0x00}}, {1, [3]byte{0x29, 0x00, 0x00}},
{1, [3]byte{0x3b, 0x00, 0x00}}, {2, [3]byte{0xc2, 0xac, 0x00}},
{1, [3]byte{0x2d, 0x00, 0x00}}, {1, [3]byte{0x2f, 0x00, 0x00}},
{2, [3]byte{0xc3, 0x82, 0x00}}, {2, [3]byte{0xc3, 0x84, 0x00}},
{2, [3]byte{0xc3, 0x80, 0x00}}, {2, [3]byte{0xc3, 0x81, 0x00}},
{2, [3]byte{0xc3, 0x83, 0x00}}, {2, [3]byte{0xc3, 0x85, 0x00}},
{2, [3]byte{0xc3, 0x87, 0x00}}, {2, [3]byte{0xc3, 0x91, 0x00}},
{2, [3]byte{0xc2, 0xa6, 0x00}}, {1, [3]byte{0x2c, 0x00, 0x00}},
{1, [3]byte{0x25, 0x00, 0x00}}, {1, [3]byte{0x5f, 0x00, 0x00}},
{1, [3]byte{0x3e, 0x00, 0x00}}, {1, [3]byte{0x3f, 0x00, 0x00}},
{2, [3]byte{0xc3, 0xb8, 0x00}}, {2, [3]byte{0xc3, 0x89, 0x00}},
{2, [3]byte{0xc3, 0x8a, 0x00}}, {2, [3]byte{0xc3, 0x8b, 0x00}},
{2, [3]byte{0xc3, 0x88, 0x00}}, {2, [3]byte{0xc3, 0x8d, 0x00}},
{2, [3]byte{0xc3, 0x8e, 0x00}}, {2, [3]byte{0xc3, 0x8f, 0x00}},
{2, [3]byte{0xc3, 0x8c, 0x00}}, {1, [3]byte{0x60, 0x00, 0x00}},
{1, [3]byte{0x3a, 0x00, 0x00}}, {1, [3]byte{0x23, 0x00, 0x00}},
{1, [3]byte{0x40, 0x00, 0x00}}, {1, [3]byte{0x27, 0x00, 0x00}},
{1, [3]byte{0x3d, 0x00, 0x00}}, {1, [3]byte{0x22, 0x00, 0x00}},
{2, [3]byte{0xc3, 0x98, 0x00}}, {1, [3]byte{0x61, 0x00, 0x00}},
{1, [3]byte{0x62, 0x00, 0x00}}, {1, [3]byte{0x63, 0x00, 0x00}},
{1, [3]byte{0x64, 0x00, 0x00}}, {1, [3]byte{0x65, 0x00, 0x00}},
{1, [3]byte{0x66, 0x00, 0x00}}, {1, [3]byte{0x67, 0x00, 0x00}},
{1, [3]byte{0x68, 0x00, 0x00}}, {1, [3]byte{0x69, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xab, 0x00}}, {2, [3]byte{0xc2, 0xbb, 0x00}},
{2, [3]byte{0xc3, 0xb0, 0x00}}, {2, [3]byte{0xc3, 0xbd, 0x00}},
{2, [3]byte{0xc3, 0xbe, 0x00}}, {2, [3]byte{0xc2, 0xb1, 0x00}},
{2, [3]byte{0xc2, 0xb0, 0x00}}, {1, [3]byte{0x6a, 0x00, 0x00}},
{1, [3]byte{0x6b, 0x00, 0x00}}, {1, [3]byte{0x6c, 0x00, 0x00}},
{1, [3]byte{0x6d, 0x00, 0x00}}, {1, [3]byte{0x6e, 0x00, 0x00}},
{1, [3]byte{0x6f, 0x00, 0x00}}, {1, [3]byte{0x70, 0x00, 0x00}},
{1, [3]byte{0x71, 0x00, 0x00}}, {1, [3]byte{0x72, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xaa, 0x00}}, {2, [3]byte{0xc2, 0xba, 0x00}},
{2, [3]byte{0xc3, 0xa6, 0x00}}, {2, [3]byte{0xc2, 0xb8, 0x00}},
{2, [3]byte{0xc3, 0x86, 0x00}}, {2, [3]byte{0xc2, 0xa4, 0x00}},
{2, [3]byte{0xc2, 0xb5, 0x00}}, {1, [3]byte{0x7e, 0x00, 0x00}},
{1, [3]byte{0x73, 0x00, 0x00}}, {1, [3]byte{0x74, 0x00, 0x00}},
{1, [3]byte{0x75, 0x00, 0x00}}, {1, [3]byte{0x76, 0x00, 0x00}},
{1, [3]byte{0x77, 0x00, 0x00}}, {1, [3]byte{0x78, 0x00, 0x00}},
{1, [3]byte{0x79, 0x00, 0x00}}, {1, [3]byte{0x7a, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xa1, 0x00}}, {2, [3]byte{0xc2, 0xbf, 0x00}},
{2, [3]byte{0xc3, 0x90, 0x00}}, {2, [3]byte{0xc3, 0x9d, 0x00}},
{2, [3]byte{0xc3, 0x9e, 0x00}}, {2, [3]byte{0xc2, 0xae, 0x00}},
{1, [3]byte{0x5e, 0x00, 0x00}}, {2, [3]byte{0xc2, 0xa3, 0x00}},
{2, [3]byte{0xc2, 0xa5, 0x00}}, {2, [3]byte{0xc2, 0xb7, 0x00}},
{2, [3]byte{0xc2, 0xa9, 0x00}}, {2, [3]byte{0xc2, 0xa7, 0x00}},
{2, [3]byte{0xc2, 0xb6, 0x00}}, {2, [3]byte{0xc2, 0xbc, 0x00}},
{2, [3]byte{0xc2, 0xbd, 0x00}}, {2, [3]byte{0xc2, 0xbe, 0x00}},
{1, [3]byte{0x5b, 0x00, 0x00}}, {1, [3]byte{0x5d, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xaf, 0x00}}, {2, [3]byte{0xc2, 0xa8, 0x00}},
{2, [3]byte{0xc2, 0xb4, 0x00}}, {2, [3]byte{0xc3, 0x97, 0x00}},
{1, [3]byte{0x7b, 0x00, 0x00}}, {1, [3]byte{0x41, 0x00, 0x00}},
{1, [3]byte{0x42, 0x00, 0x00}}, {1, [3]byte{0x43, 0x00, 0x00}},
{1, [3]byte{0x44, 0x00, 0x00}}, {1, [3]byte{0x45, 0x00, 0x00}},
{1, [3]byte{0x46, 0x00, 0x00}}, {1, [3]byte{0x47, 0x00, 0x00}},
{1, [3]byte{0x48, 0x00, 0x00}}, {1, [3]byte{0x49, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xad, 0x00}}, {2, [3]byte{0xc3, 0xb4, 0x00}},
{2, [3]byte{0xc3, 0xb6, 0x00}}, {2, [3]byte{0xc3, 0xb2, 0x00}},
{2, [3]byte{0xc3, 0xb3, 0x00}}, {2, [3]byte{0xc3, 0xb5, 0x00}},
{1, [3]byte{0x7d, 0x00, 0x00}}, {1, [3]byte{0x4a, 0x00, 0x00}},
{1, [3]byte{0x4b, 0x00, 0x00}}, {1, [3]byte{0x4c, 0x00, 0x00}},
{1, [3]byte{0x4d, 0x00, 0x00}}, {1, [3]byte{0x4e, 0x00, 0x00}},
{1, [3]byte{0x4f, 0x00, 0x00}}, {1, [3]byte{0x50, 0x00, 0x00}},
{1, [3]byte{0x51, 0x00, 0x00}}, {1, [3]byte{0x52, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xb9, 0x00}}, {2, [3]byte{0xc3, 0xbb, 0x00}},
{2, [3]byte{0xc3, 0xbc, 0x00}}, {2, [3]byte{0xc3, 0xb9, 0x00}},
{2, [3]byte{0xc3, 0xba, 0x00}}, {2, [3]byte{0xc3, 0xbf, 0x00}},
{1, [3]byte{0x5c, 0x00, 0x00}}, {2, [3]byte{0xc3, 0xb7, 0x00}},
{1, [3]byte{0x53, 0x00, 0x00}}, {1, [3]byte{0x54, 0x00, 0x00}},
{1, [3]byte{0x55, 0x00, 0x00}}, {1, [3]byte{0x56, 0x00, 0x00}},
{1, [3]byte{0x57, 0x00, 0x00}}, {1, [3]byte{0x58, 0x00, 0x00}},
{1, [3]byte{0x59, 0x00, 0x00}}, {1, [3]byte{0x5a, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xb2, 0x00}}, {2, [3]byte{0xc3, 0x94, 0x00}},
{2, [3]byte{0xc3, 0x96, 0x00}}, {2, [3]byte{0xc3, 0x92, 0x00}},
{2, [3]byte{0xc3, 0x93, 0x00}}, {2, [3]byte{0xc3, 0x95, 0x00}},
{1, [3]byte{0x30, 0x00, 0x00}}, {1, [3]byte{0x31, 0x00, 0x00}},
{1, [3]byte{0x32, 0x00, 0x00}}, {1, [3]byte{0x33, 0x00, 0x00}},
{1, [3]byte{0x34, 0x00, 0x00}}, {1, [3]byte{0x35, 0x00, 0x00}},
{1, [3]byte{0x36, 0x00, 0x00}}, {1, [3]byte{0x37, 0x00, 0x00}},
{1, [3]byte{0x38, 0x00, 0x00}}, {1, [3]byte{0x39, 0x00, 0x00}},
{2, [3]byte{0xc2, 0xb3, 0x00}}, {2, [3]byte{0xc3, 0x9b, 0x00}},
{2, [3]byte{0xc3, 0x9c, 0x00}}, {2, [3]byte{0xc3, 0x99, 0x00}},
{2, [3]byte{0xc3, 0x9a, 0x00}}, {2, [3]byte{0xc2, 0x9f, 0x00}},
},
encode: [256]uint32{
0x00000000, 0x01000001, 0x02000002, 0x03000003, 0x37000004, 0x2d000005, 0x2e000006, 0x2f000007,
0x16000008, 0x05000009, 0x2500000a, 0x0b00000b, 0x0c00000c, 0x0d00000d, 0x0e00000e, 0x0f00000f,
0x10000010, 0x11000011, 0x12000012, 0x13000013, 0x3c000014, 0x3d000015, 0x32000016, 0x26000017,
0x18000018, 0x19000019, 0x3f00001a, 0x2700001b, 0x1c00001c, 0x1d00001d, 0x1e00001e, 0x1f00001f,
0x40000020, 0x5a000021, 0x7f000022, 0x7b000023, 0x5b000024, 0x6c000025, 0x50000026, 0x7d000027,
0x4d000028, 0x5d000029, 0x5c00002a, 0x4e00002b, 0x6b00002c, 0x6000002d, 0x4b00002e, 0x6100002f,
0xf0000030, 0xf1000031, 0xf2000032, 0xf3000033, 0xf4000034, 0xf5000035, 0xf6000036, 0xf7000037,
0xf8000038, 0xf9000039, 0x7a00003a, 0x5e00003b, 0x4c00003c, 0x7e00003d, 0x6e00003e, 0x6f00003f,
0x7c000040, 0xc1000041, 0xc2000042, 0xc3000043, 0xc4000044, 0xc5000045, 0xc6000046, 0xc7000047,
0xc8000048, 0xc9000049, 0xd100004a, 0xd200004b, 0xd300004c, 0xd400004d, 0xd500004e, 0xd600004f,
0xd7000050, 0xd8000051, 0xd9000052, 0xe2000053, 0xe3000054, 0xe4000055, 0xe5000056, 0xe6000057,
0xe7000058, 0xe8000059, 0xe900005a, 0xba00005b, 0xe000005c, 0xbb00005d, 0xb000005e, 0x6d00005f,
0x79000060, 0x81000061, 0x82000062, 0x83000063, 0x84000064, 0x85000065, 0x86000066, 0x87000067,
0x88000068, 0x89000069, 0x9100006a, 0x9200006b, 0x9300006c, 0x9400006d, 0x9500006e, 0x9600006f,
0x97000070, 0x98000071, 0x99000072, 0xa2000073, 0xa3000074, 0xa4000075, 0xa5000076, 0xa6000077,
0xa7000078, 0xa8000079, 0xa900007a, 0xc000007b, 0x4f00007c, 0xd000007d, 0xa100007e, 0x0700007f,
0x20000080, 0x21000081, 0x22000082, 0x23000083, 0x24000084, 0x15000085, 0x06000086, 0x17000087,
0x28000088, 0x29000089, 0x2a00008a, 0x2b00008b, 0x2c00008c, 0x0900008d, 0x0a00008e, 0x1b00008f,
0x30000090, 0x31000091, 0x1a000092, 0x33000093, 0x34000094, 0x35000095, 0x36000096, 0x08000097,
0x38000098, 0x39000099, 0x3a00009a, 0x3b00009b, 0x0400009c, 0x1400009d, 0x3e00009e, 0xff00009f,
0x410000a0, 0xaa0000a1, 0x4a0000a2, 0xb10000a3, 0x9f0000a4, 0xb20000a5, 0x6a0000a6, 0xb50000a7,
0xbd0000a8, 0xb40000a9, 0x9a0000aa, 0x8a0000ab, 0x5f0000ac, 0xca0000ad, 0xaf0000ae, 0xbc0000af,
0x900000b0, 0x8f0000b1, 0xea0000b2, 0xfa0000b3, 0xbe0000b4, 0xa00000b5, 0xb60000b6, 0xb30000b7,
0x9d0000b8, 0xda0000b9, 0x9b0000ba, 0x8b0000bb, 0xb70000bc, 0xb80000bd, 0xb90000be, 0xab0000bf,
0x640000c0, 0x650000c1, 0x620000c2, 0x660000c3, 0x630000c4, 0x670000c5, 0x9e0000c6, 0x680000c7,
0x740000c8, 0x710000c9, 0x720000ca, 0x730000cb, 0x780000cc, 0x750000cd, 0x760000ce, 0x770000cf,
0xac0000d0, 0x690000d1, 0xed0000d2, 0xee0000d3, 0xeb0000d4, 0xef0000d5, 0xec0000d6, 0xbf0000d7,
0x800000d8, 0xfd0000d9, 0xfe0000da, 0xfb0000db, 0xfc0000dc, 0xad0000dd, 0xae0000de, 0x590000df,
0x440000e0, 0x450000e1, 0x420000e2, 0x460000e3, 0x430000e4, 0x470000e5, 0x9c0000e6, 0x480000e7,
0x540000e8, 0x510000e9, 0x520000ea, 0x530000eb, 0x580000ec, 0x550000ed, 0x560000ee, 0x570000ef,
0x8c0000f0, 0x490000f1, 0xcd0000f2, 0xce0000f3, 0xcb0000f4, 0xcf0000f5, 0xcc0000f6, 0xe10000f7,
0x700000f8, 0xdd0000f9, 0xde0000fa, 0xdb0000fb, 0xdc0000fc, 0x8d0000fd, 0x8e0000fe, 0xdf0000ff,
},
}
// CodePage437 is the IBM Code Page 437 encoding.
var CodePage437 *Charmap = &codePage437
var CodePage437 encoding.Encoding = &codePage437
var codePage437 = Charmap{
var codePage437 = charmap{
name: "IBM Code Page 437",
mib: identifier.PC8CodePage437,
asciiSuperset: true,
@ -358,9 +183,9 @@ var codePage437 = Charmap{
}
// CodePage850 is the IBM Code Page 850 encoding.
var CodePage850 *Charmap = &codePage850
var CodePage850 encoding.Encoding = &codePage850
var codePage850 = Charmap{
var codePage850 = charmap{
name: "IBM Code Page 850",
mib: identifier.PC850Multilingual,
asciiSuperset: true,
@ -533,9 +358,9 @@ var codePage850 = Charmap{
}
// CodePage852 is the IBM Code Page 852 encoding.
var CodePage852 *Charmap = &codePage852
var CodePage852 encoding.Encoding = &codePage852
var codePage852 = Charmap{
var codePage852 = charmap{
name: "IBM Code Page 852",
mib: identifier.PCp852,
asciiSuperset: true,
@ -708,9 +533,9 @@ var codePage852 = Charmap{
}
// CodePage855 is the IBM Code Page 855 encoding.
var CodePage855 *Charmap = &codePage855
var CodePage855 encoding.Encoding = &codePage855
var codePage855 = Charmap{
var codePage855 = charmap{
name: "IBM Code Page 855",
mib: identifier.IBM855,
asciiSuperset: true,
@ -883,9 +708,9 @@ var codePage855 = Charmap{
}
// CodePage858 is the Windows Code Page 858 encoding.
var CodePage858 *Charmap = &codePage858
var CodePage858 encoding.Encoding = &codePage858
var codePage858 = Charmap{
var codePage858 = charmap{
name: "Windows Code Page 858",
mib: identifier.IBM00858,
asciiSuperset: true,
@ -1058,9 +883,9 @@ var codePage858 = Charmap{
}
// CodePage860 is the IBM Code Page 860 encoding.
var CodePage860 *Charmap = &codePage860
var CodePage860 encoding.Encoding = &codePage860
var codePage860 = Charmap{
var codePage860 = charmap{
name: "IBM Code Page 860",
mib: identifier.IBM860,
asciiSuperset: true,
@ -1233,9 +1058,9 @@ var codePage860 = Charmap{
}
// CodePage862 is the IBM Code Page 862 encoding.
var CodePage862 *Charmap = &codePage862
var CodePage862 encoding.Encoding = &codePage862
var codePage862 = Charmap{
var codePage862 = charmap{
name: "IBM Code Page 862",
mib: identifier.PC862LatinHebrew,
asciiSuperset: true,
@ -1408,9 +1233,9 @@ var codePage862 = Charmap{
}
// CodePage863 is the IBM Code Page 863 encoding.
var CodePage863 *Charmap = &codePage863
var CodePage863 encoding.Encoding = &codePage863
var codePage863 = Charmap{
var codePage863 = charmap{
name: "IBM Code Page 863",
mib: identifier.IBM863,
asciiSuperset: true,
@ -1583,9 +1408,9 @@ var codePage863 = Charmap{
}
// CodePage865 is the IBM Code Page 865 encoding.
var CodePage865 *Charmap = &codePage865
var CodePage865 encoding.Encoding = &codePage865
var codePage865 = Charmap{
var codePage865 = charmap{
name: "IBM Code Page 865",
mib: identifier.IBM865,
asciiSuperset: true,
@ -1758,9 +1583,9 @@ var codePage865 = Charmap{
}
// CodePage866 is the IBM Code Page 866 encoding.
var CodePage866 *Charmap = &codePage866
var CodePage866 encoding.Encoding = &codePage866
var codePage866 = Charmap{
var codePage866 = charmap{
name: "IBM Code Page 866",
mib: identifier.IBM866,
asciiSuperset: true,
@ -1933,9 +1758,9 @@ var codePage866 = Charmap{
}
// CodePage1047 is the IBM Code Page 1047 encoding.
var CodePage1047 *Charmap = &codePage1047
var CodePage1047 encoding.Encoding = &codePage1047
var codePage1047 = Charmap{
var codePage1047 = charmap{
name: "IBM Code Page 1047",
mib: identifier.IBM1047,
asciiSuperset: false,
@ -2108,9 +1933,9 @@ var codePage1047 = Charmap{
}
// CodePage1140 is the IBM Code Page 1140 encoding.
var CodePage1140 *Charmap = &codePage1140
var CodePage1140 encoding.Encoding = &codePage1140
var codePage1140 = Charmap{
var codePage1140 = charmap{
name: "IBM Code Page 1140",
mib: identifier.IBM01140,
asciiSuperset: false,
@ -2283,9 +2108,9 @@ var codePage1140 = Charmap{
}
// ISO8859_1 is the ISO 8859-1 encoding.
var ISO8859_1 *Charmap = &iso8859_1
var ISO8859_1 encoding.Encoding = &iso8859_1
var iso8859_1 = Charmap{
var iso8859_1 = charmap{
name: "ISO 8859-1",
mib: identifier.ISOLatin1,
asciiSuperset: true,
@ -2458,9 +2283,9 @@ var iso8859_1 = Charmap{
}
// ISO8859_2 is the ISO 8859-2 encoding.
var ISO8859_2 *Charmap = &iso8859_2
var ISO8859_2 encoding.Encoding = &iso8859_2
var iso8859_2 = Charmap{
var iso8859_2 = charmap{
name: "ISO 8859-2",
mib: identifier.ISOLatin2,
asciiSuperset: true,
@ -2633,9 +2458,9 @@ var iso8859_2 = Charmap{
}
// ISO8859_3 is the ISO 8859-3 encoding.
var ISO8859_3 *Charmap = &iso8859_3
var ISO8859_3 encoding.Encoding = &iso8859_3
var iso8859_3 = Charmap{
var iso8859_3 = charmap{
name: "ISO 8859-3",
mib: identifier.ISOLatin3,
asciiSuperset: true,
@ -2808,9 +2633,9 @@ var iso8859_3 = Charmap{
}
// ISO8859_4 is the ISO 8859-4 encoding.
var ISO8859_4 *Charmap = &iso8859_4
var ISO8859_4 encoding.Encoding = &iso8859_4
var iso8859_4 = Charmap{
var iso8859_4 = charmap{
name: "ISO 8859-4",
mib: identifier.ISOLatin4,
asciiSuperset: true,
@ -2983,9 +2808,9 @@ var iso8859_4 = Charmap{
}
// ISO8859_5 is the ISO 8859-5 encoding.
var ISO8859_5 *Charmap = &iso8859_5
var ISO8859_5 encoding.Encoding = &iso8859_5
var iso8859_5 = Charmap{
var iso8859_5 = charmap{
name: "ISO 8859-5",
mib: identifier.ISOLatinCyrillic,
asciiSuperset: true,
@ -3158,9 +2983,9 @@ var iso8859_5 = Charmap{
}
// ISO8859_6 is the ISO 8859-6 encoding.
var ISO8859_6 *Charmap = &iso8859_6
var ISO8859_6 encoding.Encoding = &iso8859_6
var iso8859_6 = Charmap{
var iso8859_6 = charmap{
name: "ISO 8859-6",
mib: identifier.ISOLatinArabic,
asciiSuperset: true,
@ -3333,9 +3158,9 @@ var iso8859_6 = Charmap{
}
// ISO8859_7 is the ISO 8859-7 encoding.
var ISO8859_7 *Charmap = &iso8859_7
var ISO8859_7 encoding.Encoding = &iso8859_7
var iso8859_7 = Charmap{
var iso8859_7 = charmap{
name: "ISO 8859-7",
mib: identifier.ISOLatinGreek,
asciiSuperset: true,
@ -3508,9 +3333,9 @@ var iso8859_7 = Charmap{
}
// ISO8859_8 is the ISO 8859-8 encoding.
var ISO8859_8 *Charmap = &iso8859_8
var ISO8859_8 encoding.Encoding = &iso8859_8
var iso8859_8 = Charmap{
var iso8859_8 = charmap{
name: "ISO 8859-8",
mib: identifier.ISOLatinHebrew,
asciiSuperset: true,
@ -3683,9 +3508,9 @@ var iso8859_8 = Charmap{
}
// ISO8859_9 is the ISO 8859-9 encoding.
var ISO8859_9 *Charmap = &iso8859_9
var ISO8859_9 encoding.Encoding = &iso8859_9
var iso8859_9 = Charmap{
var iso8859_9 = charmap{
name: "ISO 8859-9",
mib: identifier.ISOLatin5,
asciiSuperset: true,
@ -3858,9 +3683,9 @@ var iso8859_9 = Charmap{
}
// ISO8859_10 is the ISO 8859-10 encoding.
var ISO8859_10 *Charmap = &iso8859_10
var ISO8859_10 encoding.Encoding = &iso8859_10
var iso8859_10 = Charmap{
var iso8859_10 = charmap{
name: "ISO 8859-10",
mib: identifier.ISOLatin6,
asciiSuperset: true,
@ -4033,9 +3858,9 @@ var iso8859_10 = Charmap{
}
// ISO8859_13 is the ISO 8859-13 encoding.
var ISO8859_13 *Charmap = &iso8859_13
var ISO8859_13 encoding.Encoding = &iso8859_13
var iso8859_13 = Charmap{
var iso8859_13 = charmap{
name: "ISO 8859-13",
mib: identifier.ISO885913,
asciiSuperset: true,
@ -4208,9 +4033,9 @@ var iso8859_13 = Charmap{
}
// ISO8859_14 is the ISO 8859-14 encoding.
var ISO8859_14 *Charmap = &iso8859_14
var ISO8859_14 encoding.Encoding = &iso8859_14
var iso8859_14 = Charmap{
var iso8859_14 = charmap{
name: "ISO 8859-14",
mib: identifier.ISO885914,
asciiSuperset: true,
@ -4383,9 +4208,9 @@ var iso8859_14 = Charmap{
}
// ISO8859_15 is the ISO 8859-15 encoding.
var ISO8859_15 *Charmap = &iso8859_15
var ISO8859_15 encoding.Encoding = &iso8859_15
var iso8859_15 = Charmap{
var iso8859_15 = charmap{
name: "ISO 8859-15",
mib: identifier.ISO885915,
asciiSuperset: true,
@ -4558,9 +4383,9 @@ var iso8859_15 = Charmap{
}
// ISO8859_16 is the ISO 8859-16 encoding.
var ISO8859_16 *Charmap = &iso8859_16
var ISO8859_16 encoding.Encoding = &iso8859_16
var iso8859_16 = Charmap{
var iso8859_16 = charmap{
name: "ISO 8859-16",
mib: identifier.ISO885916,
asciiSuperset: true,
@ -4733,9 +4558,9 @@ var iso8859_16 = Charmap{
}
// KOI8R is the KOI8-R encoding.
var KOI8R *Charmap = &koi8R
var KOI8R encoding.Encoding = &koi8R
var koi8R = Charmap{
var koi8R = charmap{
name: "KOI8-R",
mib: identifier.KOI8R,
asciiSuperset: true,
@ -4908,9 +4733,9 @@ var koi8R = Charmap{
}
// KOI8U is the KOI8-U encoding.
var KOI8U *Charmap = &koi8U
var KOI8U encoding.Encoding = &koi8U
var koi8U = Charmap{
var koi8U = charmap{
name: "KOI8-U",
mib: identifier.KOI8U,
asciiSuperset: true,
@ -5083,9 +4908,9 @@ var koi8U = Charmap{
}
// Macintosh is the Macintosh encoding.
var Macintosh *Charmap = &macintosh
var Macintosh encoding.Encoding = &macintosh
var macintosh = Charmap{
var macintosh = charmap{
name: "Macintosh",
mib: identifier.Macintosh,
asciiSuperset: true,
@ -5258,9 +5083,9 @@ var macintosh = Charmap{
}
// MacintoshCyrillic is the Macintosh Cyrillic encoding.
var MacintoshCyrillic *Charmap = &macintoshCyrillic
var MacintoshCyrillic encoding.Encoding = &macintoshCyrillic
var macintoshCyrillic = Charmap{
var macintoshCyrillic = charmap{
name: "Macintosh Cyrillic",
mib: identifier.MacintoshCyrillic,
asciiSuperset: true,
@ -5433,9 +5258,9 @@ var macintoshCyrillic = Charmap{
}
// Windows874 is the Windows 874 encoding.
var Windows874 *Charmap = &windows874
var Windows874 encoding.Encoding = &windows874
var windows874 = Charmap{
var windows874 = charmap{
name: "Windows 874",
mib: identifier.Windows874,
asciiSuperset: true,
@ -5608,9 +5433,9 @@ var windows874 = Charmap{
}
// Windows1250 is the Windows 1250 encoding.
var Windows1250 *Charmap = &windows1250
var Windows1250 encoding.Encoding = &windows1250
var windows1250 = Charmap{
var windows1250 = charmap{
name: "Windows 1250",
mib: identifier.Windows1250,
asciiSuperset: true,
@ -5783,9 +5608,9 @@ var windows1250 = Charmap{
}
// Windows1251 is the Windows 1251 encoding.
var Windows1251 *Charmap = &windows1251
var Windows1251 encoding.Encoding = &windows1251
var windows1251 = Charmap{
var windows1251 = charmap{
name: "Windows 1251",
mib: identifier.Windows1251,
asciiSuperset: true,
@ -5958,9 +5783,9 @@ var windows1251 = Charmap{
}
// Windows1252 is the Windows 1252 encoding.
var Windows1252 *Charmap = &windows1252
var Windows1252 encoding.Encoding = &windows1252
var windows1252 = Charmap{
var windows1252 = charmap{
name: "Windows 1252",
mib: identifier.Windows1252,
asciiSuperset: true,
@ -6133,9 +5958,9 @@ var windows1252 = Charmap{
}
// Windows1253 is the Windows 1253 encoding.
var Windows1253 *Charmap = &windows1253
var Windows1253 encoding.Encoding = &windows1253
var windows1253 = Charmap{
var windows1253 = charmap{
name: "Windows 1253",
mib: identifier.Windows1253,
asciiSuperset: true,
@ -6308,9 +6133,9 @@ var windows1253 = Charmap{
}
// Windows1254 is the Windows 1254 encoding.
var Windows1254 *Charmap = &windows1254
var Windows1254 encoding.Encoding = &windows1254
var windows1254 = Charmap{
var windows1254 = charmap{
name: "Windows 1254",
mib: identifier.Windows1254,
asciiSuperset: true,
@ -6483,9 +6308,9 @@ var windows1254 = Charmap{
}
// Windows1255 is the Windows 1255 encoding.
var Windows1255 *Charmap = &windows1255
var Windows1255 encoding.Encoding = &windows1255
var windows1255 = Charmap{
var windows1255 = charmap{
name: "Windows 1255",
mib: identifier.Windows1255,
asciiSuperset: true,
@ -6593,7 +6418,7 @@ var windows1255 = Charmap{
{2, [3]byte{0xd6, 0xb4, 0x00}}, {2, [3]byte{0xd6, 0xb5, 0x00}},
{2, [3]byte{0xd6, 0xb6, 0x00}}, {2, [3]byte{0xd6, 0xb7, 0x00}},
{2, [3]byte{0xd6, 0xb8, 0x00}}, {2, [3]byte{0xd6, 0xb9, 0x00}},
{2, [3]byte{0xd6, 0xba, 0x00}}, {2, [3]byte{0xd6, 0xbb, 0x00}},
{3, [3]byte{0xef, 0xbf, 0xbd}}, {2, [3]byte{0xd6, 0xbb, 0x00}},
{2, [3]byte{0xd6, 0xbc, 0x00}}, {2, [3]byte{0xd6, 0xbd, 0x00}},
{2, [3]byte{0xd6, 0xbe, 0x00}}, {2, [3]byte{0xd6, 0xbf, 0x00}},
{2, [3]byte{0xd7, 0x80, 0x00}}, {2, [3]byte{0xd7, 0x81, 0x00}},
@ -6643,24 +6468,24 @@ var windows1255 = Charmap{
0xb20000b2, 0xb30000b3, 0xb40000b4, 0xb50000b5, 0xb60000b6, 0xb70000b7, 0xb80000b8, 0xb90000b9,
0xbb0000bb, 0xbc0000bc, 0xbd0000bd, 0xbe0000be, 0xbf0000bf, 0xaa0000d7, 0xba0000f7, 0x83000192,
0x880002c6, 0x980002dc, 0xc00005b0, 0xc10005b1, 0xc20005b2, 0xc30005b3, 0xc40005b4, 0xc50005b5,
0xc60005b6, 0xc70005b7, 0xc80005b8, 0xc90005b9, 0xca0005ba, 0xcb0005bb, 0xcc0005bc, 0xcd0005bd,
0xce0005be, 0xcf0005bf, 0xd00005c0, 0xd10005c1, 0xd20005c2, 0xd30005c3, 0xe00005d0, 0xe10005d1,
0xe20005d2, 0xe30005d3, 0xe40005d4, 0xe50005d5, 0xe60005d6, 0xe70005d7, 0xe80005d8, 0xe90005d9,
0xea0005da, 0xeb0005db, 0xec0005dc, 0xed0005dd, 0xee0005de, 0xef0005df, 0xf00005e0, 0xf10005e1,
0xf20005e2, 0xf30005e3, 0xf40005e4, 0xf50005e5, 0xf60005e6, 0xf70005e7, 0xf80005e8, 0xf90005e9,
0xfa0005ea, 0xd40005f0, 0xd50005f1, 0xd60005f2, 0xd70005f3, 0xd80005f4, 0xfd00200e, 0xfe00200f,
0x96002013, 0x97002014, 0x91002018, 0x92002019, 0x8200201a, 0x9300201c, 0x9400201d, 0x8400201e,
0x86002020, 0x87002021, 0x95002022, 0x85002026, 0x89002030, 0x8b002039, 0x9b00203a, 0xa40020aa,
0x800020ac, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122,
0xc60005b6, 0xc70005b7, 0xc80005b8, 0xc90005b9, 0xcb0005bb, 0xcc0005bc, 0xcd0005bd, 0xce0005be,
0xcf0005bf, 0xd00005c0, 0xd10005c1, 0xd20005c2, 0xd30005c3, 0xe00005d0, 0xe10005d1, 0xe20005d2,
0xe30005d3, 0xe40005d4, 0xe50005d5, 0xe60005d6, 0xe70005d7, 0xe80005d8, 0xe90005d9, 0xea0005da,
0xeb0005db, 0xec0005dc, 0xed0005dd, 0xee0005de, 0xef0005df, 0xf00005e0, 0xf10005e1, 0xf20005e2,
0xf30005e3, 0xf40005e4, 0xf50005e5, 0xf60005e6, 0xf70005e7, 0xf80005e8, 0xf90005e9, 0xfa0005ea,
0xd40005f0, 0xd50005f1, 0xd60005f2, 0xd70005f3, 0xd80005f4, 0xfd00200e, 0xfe00200f, 0x96002013,
0x97002014, 0x91002018, 0x92002019, 0x8200201a, 0x9300201c, 0x9400201d, 0x8400201e, 0x86002020,
0x87002021, 0x95002022, 0x85002026, 0x89002030, 0x8b002039, 0x9b00203a, 0xa40020aa, 0x800020ac,
0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122,
0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122,
0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122, 0x99002122,
},
}
// Windows1256 is the Windows 1256 encoding.
var Windows1256 *Charmap = &windows1256
var Windows1256 encoding.Encoding = &windows1256
var windows1256 = Charmap{
var windows1256 = charmap{
name: "Windows 1256",
mib: identifier.Windows1256,
asciiSuperset: true,
@ -6833,9 +6658,9 @@ var windows1256 = Charmap{
}
// Windows1257 is the Windows 1257 encoding.
var Windows1257 *Charmap = &windows1257
var Windows1257 encoding.Encoding = &windows1257
var windows1257 = Charmap{
var windows1257 = charmap{
name: "Windows 1257",
mib: identifier.Windows1257,
asciiSuperset: true,
@ -7008,9 +6833,9 @@ var windows1257 = Charmap{
}
// Windows1258 is the Windows 1258 encoding.
var Windows1258 *Charmap = &windows1258
var Windows1258 encoding.Encoding = &windows1258
var windows1258 = Charmap{
var windows1258 = charmap{
name: "Windows 1258",
mib: identifier.Windows1258,
asciiSuperset: true,
@ -7185,9 +7010,9 @@ var windows1258 = Charmap{
// XUserDefined is the X-User-Defined encoding.
//
// It is defined at http://encoding.spec.whatwg.org/#x-user-defined
var XUserDefined *Charmap = &xUserDefined
var XUserDefined encoding.Encoding = &xUserDefined
var xUserDefined = Charmap{
var xUserDefined = charmap{
name: "X-User-Defined",
mib: identifier.XUserDefined,
asciiSuperset: true,
@ -7359,7 +7184,6 @@ var xUserDefined = Charmap{
},
}
var listAll = []encoding.Encoding{
CodePage037,
CodePage437,
CodePage850,
CodePage852,
@ -7407,4 +7231,4 @@ var listAll = []encoding.Encoding{
XUserDefined,
}
// Total table size 87024 bytes (84KiB); checksum: 811C9DC5
// Total table size 84952 bytes (82KiB); checksum: 811C9DC5

View file

@ -52,7 +52,7 @@ type Decoder struct {
}
// Bytes converts the given encoded bytes to UTF-8. It returns the converted
// bytes or 0, err if any error occurred.
// bytes or nil, err if any error occurred.
func (d *Decoder) Bytes(b []byte) ([]byte, error) {
b, _, err := transform.Bytes(d, b)
if err != nil {
@ -62,7 +62,7 @@ func (d *Decoder) Bytes(b []byte) ([]byte, error) {
}
// String converts the given encoded string to UTF-8. It returns the converted
// string or 0, err if any error occurred.
// string or "", err if any error occurred.
func (d *Decoder) String(s string) (string, error) {
s, _, err := transform.String(d, s)
if err != nil {
@ -95,7 +95,7 @@ type Encoder struct {
_ struct{}
}
// Bytes converts bytes from UTF-8. It returns the converted bytes or 0, err if
// Bytes converts bytes from UTF-8. It returns the converted bytes or nil, err if
// any error occurred.
func (e *Encoder) Bytes(b []byte) ([]byte, error) {
b, _, err := transform.Bytes(e, b)
@ -106,7 +106,7 @@ func (e *Encoder) Bytes(b []byte) ([]byte, error) {
}
// String converts a string from UTF-8. It returns the converted string or
// 0, err if any error occurred.
// "", err if any error occurred.
func (e *Encoder) String(s string) (string, error) {
s, _, err := transform.String(e, s)
if err != nil {

View file

@ -1,137 +0,0 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
package main
import (
"bytes"
"encoding/xml"
"fmt"
"io"
"log"
"strings"
"golang.org/x/text/internal/gen"
)
type registry struct {
XMLName xml.Name `xml:"registry"`
Updated string `xml:"updated"`
Registry []struct {
ID string `xml:"id,attr"`
Record []struct {
Name string `xml:"name"`
Xref []struct {
Type string `xml:"type,attr"`
Data string `xml:"data,attr"`
} `xml:"xref"`
Desc struct {
Data string `xml:",innerxml"`
// Any []struct {
// Data string `xml:",chardata"`
// } `xml:",any"`
// Data string `xml:",chardata"`
} `xml:"description,"`
MIB string `xml:"value"`
Alias []string `xml:"alias"`
MIME string `xml:"preferred_alias"`
} `xml:"record"`
} `xml:"registry"`
}
func main() {
r := gen.OpenIANAFile("assignments/character-sets/character-sets.xml")
reg := &registry{}
if err := xml.NewDecoder(r).Decode(&reg); err != nil && err != io.EOF {
log.Fatalf("Error decoding charset registry: %v", err)
}
if len(reg.Registry) == 0 || reg.Registry[0].ID != "character-sets-1" {
log.Fatalf("Unexpected ID %s", reg.Registry[0].ID)
}
w := &bytes.Buffer{}
fmt.Fprintf(w, "const (\n")
for _, rec := range reg.Registry[0].Record {
constName := ""
for _, a := range rec.Alias {
if strings.HasPrefix(a, "cs") && strings.IndexByte(a, '-') == -1 {
// Some of the constant definitions have comments in them. Strip those.
constName = strings.Title(strings.SplitN(a[2:], "\n", 2)[0])
}
}
if constName == "" {
switch rec.MIB {
case "2085":
constName = "HZGB2312" // Not listed as alias for some reason.
default:
log.Fatalf("No cs alias defined for %s.", rec.MIB)
}
}
if rec.MIME != "" {
rec.MIME = fmt.Sprintf(" (MIME: %s)", rec.MIME)
}
fmt.Fprintf(w, "// %s is the MIB identifier with IANA name %s%s.\n//\n", constName, rec.Name, rec.MIME)
if len(rec.Desc.Data) > 0 {
fmt.Fprint(w, "// ")
d := xml.NewDecoder(strings.NewReader(rec.Desc.Data))
inElem := true
attr := ""
for {
t, err := d.Token()
if err != nil {
if err != io.EOF {
log.Fatal(err)
}
break
}
switch x := t.(type) {
case xml.CharData:
attr = "" // Don't need attribute info.
a := bytes.Split([]byte(x), []byte("\n"))
for i, b := range a {
if b = bytes.TrimSpace(b); len(b) != 0 {
if !inElem && i > 0 {
fmt.Fprint(w, "\n// ")
}
inElem = false
fmt.Fprintf(w, "%s ", string(b))
}
}
case xml.StartElement:
if x.Name.Local == "xref" {
inElem = true
use := false
for _, a := range x.Attr {
if a.Name.Local == "type" {
use = use || a.Value != "person"
}
if a.Name.Local == "data" && use {
attr = a.Value + " "
}
}
}
case xml.EndElement:
inElem = false
fmt.Fprint(w, attr)
}
}
fmt.Fprint(w, "\n")
}
for _, x := range rec.Xref {
switch x.Type {
case "rfc":
fmt.Fprintf(w, "// Reference: %s\n", strings.ToUpper(x.Data))
case "uri":
fmt.Fprintf(w, "// Reference: %s\n", x.Data)
}
}
fmt.Fprintf(w, "%s MIB = %s\n", constName, rec.MIB)
fmt.Fprintln(w)
}
fmt.Fprintln(w, ")")
gen.WriteGoFile("mib.go", "identifier", w.Bytes())
}

View file

@ -41,20 +41,35 @@ func If(s Set, tIn, tNotIn transform.Transformer) Transformer {
if tNotIn == nil {
tNotIn = transform.Nop
}
sIn, ok := tIn.(transform.SpanningTransformer)
if !ok {
sIn = dummySpan{tIn}
}
sNotIn, ok := tNotIn.(transform.SpanningTransformer)
if !ok {
sNotIn = dummySpan{tNotIn}
}
a := &cond{
tIn: tIn,
tNotIn: tNotIn,
tIn: sIn,
tNotIn: sNotIn,
f: s.Contains,
}
a.Reset()
return Transformer{a}
}
type dummySpan struct{ transform.Transformer }
func (d dummySpan) Span(src []byte, atEOF bool) (n int, err error) {
return 0, transform.ErrEndOfSpan
}
type cond struct {
tIn, tNotIn transform.Transformer
tIn, tNotIn transform.SpanningTransformer
f func(rune) bool
check func(rune) bool // current check to perform
t transform.Transformer // current transformer to use
check func(rune) bool // current check to perform
t transform.SpanningTransformer // current transformer to use
}
// Reset implements transform.Transformer.
@ -84,6 +99,51 @@ func (t *cond) isNot(r rune) bool {
return false
}
// This implementation of Span doesn't help all too much, but it needs to be
// there to satisfy this package's Transformer interface.
// TODO: there are certainly room for improvements, though. For example, if
// t.t == transform.Nop (which will a common occurrence) it will save a bundle
// to special-case that loop.
func (t *cond) Span(src []byte, atEOF bool) (n int, err error) {
p := 0
for n < len(src) && err == nil {
// Don't process too much at a time as the Spanner that will be
// called on this block may terminate early.
const maxChunk = 4096
max := len(src)
if v := n + maxChunk; v < max {
max = v
}
atEnd := false
size := 0
current := t.t
for ; p < max; p += size {
r := rune(src[p])
if r < utf8.RuneSelf {
size = 1
} else if r, size = utf8.DecodeRune(src[p:]); size == 1 {
if !atEOF && !utf8.FullRune(src[p:]) {
err = transform.ErrShortSrc
break
}
}
if !t.check(r) {
// The next rune will be the start of a new run.
atEnd = true
break
}
}
n2, err2 := current.Span(src[n:p], atEnd || (atEOF && p == len(src)))
n += n2
if err2 != nil {
return n, err2
}
// At this point either err != nil or t.check will pass for the rune at p.
p = n + size
}
return n, err
}
func (t *cond) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
p := 0
for nSrc < len(src) && err == nil {
@ -99,9 +159,10 @@ func (t *cond) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error
size := 0
current := t.t
for ; p < max; p += size {
var r rune
r, size = utf8.DecodeRune(src[p:])
if r == utf8.RuneError && size == 1 {
r := rune(src[p])
if r < utf8.RuneSelf {
size = 1
} else if r, size = utf8.DecodeRune(src[p:]); size == 1 {
if !atEOF && !utf8.FullRune(src[p:]) {
err = transform.ErrShortSrc
break

View file

@ -46,9 +46,19 @@ func Predicate(f func(rune) bool) Set {
// Transformer implements the transform.Transformer interface.
type Transformer struct {
transform.Transformer
t transform.SpanningTransformer
}
func (t Transformer) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
return t.t.Transform(dst, src, atEOF)
}
func (t Transformer) Span(b []byte, atEOF bool) (n int, err error) {
return t.t.Span(b, atEOF)
}
func (t Transformer) Reset() { t.t.Reset() }
// Bytes returns a new byte slice with the result of converting b using t. It
// calls Reset on t. It returns nil if any error was found. This can only happen
// if an error-producing Transformer is passed to If.
@ -96,39 +106,57 @@ type remove func(r rune) bool
func (remove) Reset() {}
// Span implements transform.Spanner.
func (t remove) Span(src []byte, atEOF bool) (n int, err error) {
for r, size := rune(0), 0; n < len(src); {
if r = rune(src[n]); r < utf8.RuneSelf {
size = 1
} else if r, size = utf8.DecodeRune(src[n:]); size == 1 {
// Invalid rune.
if !atEOF && !utf8.FullRune(src[n:]) {
err = transform.ErrShortSrc
} else {
err = transform.ErrEndOfSpan
}
break
}
if t(r) {
err = transform.ErrEndOfSpan
break
}
n += size
}
return
}
// Transform implements transform.Transformer.
func (t remove) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
for r, size := rune(0), 0; nSrc < len(src); {
if r = rune(src[nSrc]); r < utf8.RuneSelf {
size = 1
} else {
r, size = utf8.DecodeRune(src[nSrc:])
if size == 1 {
// Invalid rune.
if !atEOF && !utf8.FullRune(src[nSrc:]) {
err = transform.ErrShortSrc
} else if r, size = utf8.DecodeRune(src[nSrc:]); size == 1 {
// Invalid rune.
if !atEOF && !utf8.FullRune(src[nSrc:]) {
err = transform.ErrShortSrc
break
}
// We replace illegal bytes with RuneError. Not doing so might
// otherwise turn a sequence of invalid UTF-8 into valid UTF-8.
// The resulting byte sequence may subsequently contain runes
// for which t(r) is true that were passed unnoticed.
if !t(utf8.RuneError) {
if nDst+3 > len(dst) {
err = transform.ErrShortDst
break
}
// We replace illegal bytes with RuneError. Not doing so might
// otherwise turn a sequence of invalid UTF-8 into valid UTF-8.
// The resulting byte sequence may subsequently contain runes
// for which t(r) is true that were passed unnoticed.
if !t(utf8.RuneError) {
if nDst+3 > len(dst) {
err = transform.ErrShortDst
break
}
dst[nDst+0] = runeErrorString[0]
dst[nDst+1] = runeErrorString[1]
dst[nDst+2] = runeErrorString[2]
nDst += 3
}
nSrc++
continue
dst[nDst+0] = runeErrorString[0]
dst[nDst+1] = runeErrorString[1]
dst[nDst+2] = runeErrorString[2]
nDst += 3
}
nSrc++
continue
}
if t(r) {
nSrc += size
continue
@ -157,6 +185,28 @@ type mapper func(rune) rune
func (mapper) Reset() {}
// Span implements transform.Spanner.
func (t mapper) Span(src []byte, atEOF bool) (n int, err error) {
for r, size := rune(0), 0; n < len(src); n += size {
if r = rune(src[n]); r < utf8.RuneSelf {
size = 1
} else if r, size = utf8.DecodeRune(src[n:]); size == 1 {
// Invalid rune.
if !atEOF && !utf8.FullRune(src[n:]) {
err = transform.ErrShortSrc
} else {
err = transform.ErrEndOfSpan
}
break
}
if t(r) != r {
err = transform.ErrEndOfSpan
break
}
}
return n, err
}
// Transform implements transform.Transformer.
func (t mapper) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
var replacement rune
@ -230,24 +280,51 @@ func ReplaceIllFormed() Transformer {
type replaceIllFormed struct{ transform.NopResetter }
func (t replaceIllFormed) Span(src []byte, atEOF bool) (n int, err error) {
for n < len(src) {
// ASCII fast path.
if src[n] < utf8.RuneSelf {
n++
continue
}
r, size := utf8.DecodeRune(src[n:])
// Look for a valid non-ASCII rune.
if r != utf8.RuneError || size != 1 {
n += size
continue
}
// Look for short source data.
if !atEOF && !utf8.FullRune(src[n:]) {
err = transform.ErrShortSrc
break
}
// We have an invalid rune.
err = transform.ErrEndOfSpan
break
}
return n, err
}
func (t replaceIllFormed) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
for nSrc < len(src) {
r, size := utf8.DecodeRune(src[nSrc:])
// Look for an ASCII rune.
if r < utf8.RuneSelf {
// ASCII fast path.
if r := src[nSrc]; r < utf8.RuneSelf {
if nDst == len(dst) {
err = transform.ErrShortDst
break
}
dst[nDst] = byte(r)
dst[nDst] = r
nDst++
nSrc++
continue
}
// Look for a valid non-ASCII rune.
if r != utf8.RuneError || size != 1 {
if _, size := utf8.DecodeRune(src[nSrc:]); size != 1 {
if size != copy(dst[nDst:], src[nSrc:nSrc+size]) {
err = transform.ErrShortDst
break

View file

@ -24,6 +24,10 @@ var (
// complete the transformation.
ErrShortSrc = errors.New("transform: short source buffer")
// ErrEndOfSpan means that the input and output (the transformed input)
// are not identical.
ErrEndOfSpan = errors.New("transform: input and output are not identical")
// errInconsistentByteCount means that Transform returned success (nil
// error) but also returned nSrc inconsistent with the src argument.
errInconsistentByteCount = errors.New("transform: inconsistent byte count returned")
@ -60,6 +64,41 @@ type Transformer interface {
Reset()
}
// SpanningTransformer extends the Transformer interface with a Span method
// that determines how much of the input already conforms to the Transformer.
type SpanningTransformer interface {
Transformer
// Span returns a position in src such that transforming src[:n] results in
// identical output src[:n] for these bytes. It does not necessarily return
// the largest such n. The atEOF argument tells whether src represents the
// last bytes of the input.
//
// Callers should always account for the n bytes consumed before
// considering the error err.
//
// A nil error means that all input bytes are known to be identical to the
// output produced by the Transformer. A nil error can be be returned
// regardless of whether atEOF is true. If err is nil, then then n must
// equal len(src); the converse is not necessarily true.
//
// ErrEndOfSpan means that the Transformer output may differ from the
// input after n bytes. Note that n may be len(src), meaning that the output
// would contain additional bytes after otherwise identical output.
// ErrShortSrc means that src had insufficient data to determine whether the
// remaining bytes would change. Other than the error conditions listed
// here, implementations are free to report other errors that arise.
//
// Calling Span can modify the Transformer state as a side effect. In
// effect, it does the transformation just as calling Transform would, only
// without copying to a destination buffer and only up to a point it can
// determine the input and output bytes are the same. This is obviously more
// limited than calling Transform, but can be more efficient in terms of
// copying and allocating buffers. Calls to Span and Transform may be
// interleaved.
Span(src []byte, atEOF bool) (n int, err error)
}
// NopResetter can be embedded by implementations of Transformer to add a nop
// Reset method.
type NopResetter struct{}
@ -278,6 +317,10 @@ func (nop) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
return n, n, err
}
func (nop) Span(src []byte, atEOF bool) (n int, err error) {
return len(src), nil
}
type discard struct{ NopResetter }
func (discard) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
@ -289,8 +332,8 @@ var (
// by consuming all bytes and writing nothing.
Discard Transformer = discard{}
// Nop is a Transformer that copies src to dst.
Nop Transformer = nop{}
// Nop is a SpanningTransformer that copies src to dst.
Nop SpanningTransformer = nop{}
)
// chain is a sequence of links. A chain with N Transformers has N+1 links and
@ -358,6 +401,8 @@ func (c *chain) Reset() {
}
}
// TODO: make chain use Span (is going to be fun to implement!)
// Transform applies the transformers of c in sequence.
func (c *chain) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
// Set up src and dst in the chain.
@ -448,8 +493,7 @@ func (c *chain) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err erro
return dstL.n, srcL.p, err
}
// RemoveFunc returns a Transformer that removes from the input all runes r for
// which f(r) is true. Illegal bytes in the input are replaced by RuneError.
// Deprecated: use runes.Remove instead.
func RemoveFunc(f func(r rune) bool) Transformer {
return removeF(f)
}

109
vendor/vendor.json vendored
View file

@ -38,10 +38,10 @@
"revisionTime": "2017-08-08T19:54:39Z"
},
{
"checksumSHA1": "GZj+/EV9o0GyIsZ5/S8gSy3ceKM=",
"checksumSHA1": "cksQ/Vucu9mWJrsLG1bjUvg4ao8=",
"path": "github.com/TomOnTime/utfutil",
"revision": "2eac7962c5478fc8b8123d73ef7d33f05f9cd8f0",
"revisionTime": "2016-09-09T15:53:55Z"
"revision": "1916859f26bb476f86a4c47c272f7c881f92619a",
"revisionTime": "2017-11-03T00:04:35Z"
},
{
"checksumSHA1": "irAylH0FbfVgkCcXP9U+Fkou9+U=",
@ -254,11 +254,10 @@
"revisionTime": "2017-08-30T08:48:20Z"
},
{
"checksumSHA1": "OnSTHmTPGTAwFMfqCSZj4z5BnEo=",
"origin": "github.com/aws/aws-sdk-go/vendor/github.com/go-ini/ini",
"checksumSHA1": "oSwR5RN8idtBQrpnQSy13t8ruoU=",
"path": "github.com/go-ini/ini",
"revision": "060d7da055ba6ec5ea7a31f116332fe5efa04ce0",
"revisionTime": "2015-11-04T20:00:20-05:00"
"revision": "32e4be5f41bb918afb6e37c07426e2ddbcb6647e",
"revisionTime": "2018-02-14T10:17:53Z"
},
{
"checksumSHA1": "yqF125xVSkmfLpIVGrLlfE05IUk=",
@ -279,11 +278,16 @@
"revisionTime": "2017-01-11T10:11:55Z"
},
{
"checksumSHA1": "Wbsfxh89xfnFDZzoEOzpfUV9+u8=",
"origin": "github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath",
"checksumSHA1": "uFNLkTxk8BkHTk0Jcub2lj2BX4M=",
"path": "github.com/google/uuid",
"revision": "dec09d789f3dba190787f8b4454c7d3c936fed9e",
"revisionTime": "2017-11-29T19:10:14Z"
},
{
"checksumSHA1": "blwbl9vPvRLtL5QlZgfpLvsFiZ4=",
"path": "github.com/jmespath/go-jmespath",
"revision": "3433f3ea46d9f8019119e7dd41274e112a2359a9",
"revisionTime": "2015-11-17T09:58:22-08:00"
"revision": "c2b33e8439af944379acbdd9c3a5fe0bc44bd8a5",
"revisionTime": "2018-02-06T20:15:40Z"
},
{
"checksumSHA1": "a8Ge6pE7oxux9ZMZVAlyEeGzCng=",
@ -303,10 +307,10 @@
"revisionTime": "2017-11-10T10:33:17Z"
},
{
"checksumSHA1": "9SrLPArdQmp45MGosz7IEGX46Ng=",
"checksumSHA1": "bdcCMc8DK5HBRPHBAmNDNQXmaVQ=",
"path": "github.com/miekg/dns/dnsutil",
"revision": "e4205768578dc90c2669e75a2f8a8bf77e3083a4",
"revisionTime": "2017-08-18T13:14:42Z"
"revision": "9fc4eb252eedf0ef8adc05169ce35da5e31beaba",
"revisionTime": "2017-11-10T10:33:17Z"
},
{
"checksumSHA1": "AYjG78HsyaohvBTuzCgdxU48dFE=",
@ -327,10 +331,10 @@
"revisionTime": "2018-01-09T18:53:19Z"
},
{
"checksumSHA1": "rJab1YdNhQooDiBWNnt7TLWPyBU=",
"checksumSHA1": "ynJSWoF6v+3zMnh9R0QmmG6iGV8=",
"path": "github.com/pkg/errors",
"revision": "2b3a18b5f0fb6b4f9190549597d3f962c02bc5eb",
"revisionTime": "2017-09-10T13:46:14Z"
"revision": "ff09b135c25aae272398c51a07235b90a75aa4f0",
"revisionTime": "2017-03-16T20:15:38Z"
},
{
"checksumSHA1": "02kfHQp1KLch4P2FG1OrEG/24t8=",
@ -346,7 +350,6 @@
},
{
"checksumSHA1": "9tjoj3PcsKSKiX47mnZIgoFrs4U=",
"origin": "github.com/StackExchange/dnscontrol/vendor/github.com/prasmussen/gandi-api/contact",
"path": "github.com/prasmussen/gandi-api/contact",
"revision": "58d3d42056619bb56e311c115c1c95294b5ec60b",
"revisionTime": "2018-02-24T13:22:02Z"
@ -383,28 +386,24 @@
},
{
"checksumSHA1": "lgixSmeD/e2zKVoMq3xJ+0ormhQ=",
"origin": "github.com/StackExchange/dnscontrol/vendor/github.com/prasmussen/gandi-api/live_dns/domain",
"path": "github.com/prasmussen/gandi-api/live_dns/domain",
"revision": "58d3d42056619bb56e311c115c1c95294b5ec60b",
"revisionTime": "2018-02-24T13:22:02Z"
},
{
"checksumSHA1": "YOKj2uCylMIaRNJHaOIVEns0K6c=",
"origin": "github.com/StackExchange/dnscontrol/vendor/github.com/prasmussen/gandi-api/live_dns/record",
"path": "github.com/prasmussen/gandi-api/live_dns/record",
"revision": "58d3d42056619bb56e311c115c1c95294b5ec60b",
"revisionTime": "2018-02-24T13:22:02Z"
},
{
"checksumSHA1": "5TdSD8BWzrTLi2OizfUv2zqbbzQ=",
"origin": "github.com/StackExchange/dnscontrol/vendor/github.com/prasmussen/gandi-api/live_dns/test_helpers",
"path": "github.com/prasmussen/gandi-api/live_dns/test_helpers",
"revision": "58d3d42056619bb56e311c115c1c95294b5ec60b",
"revisionTime": "2018-02-24T13:22:02Z"
},
{
"checksumSHA1": "qCClQEFkVUcQrQw6n+zWGl0JMwc=",
"origin": "github.com/StackExchange/dnscontrol/vendor/github.com/prasmussen/gandi-api/live_dns/zone",
"path": "github.com/prasmussen/gandi-api/live_dns/zone",
"revision": "58d3d42056619bb56e311c115c1c95294b5ec60b",
"revisionTime": "2018-02-24T13:22:02Z"
@ -554,10 +553,10 @@
"revisionTime": "2013-07-02T22:55:49Z"
},
{
"checksumSHA1": "7Qm/EUxEnhQgbMPWUsKrVswLUsc=",
"checksumSHA1": "H1Ne82yYzWRQ2DKBsmRd0Hwcl04=",
"path": "github.com/urfave/cli",
"revision": "7fb9c86b14e6a702a4157ccb5a863f07d844a207",
"revisionTime": "2017-09-11T04:08:19Z"
"revision": "75104e932ac2ddb944a6ea19d9f9f26316ff1145",
"revisionTime": "2018-01-06T19:10:48Z"
},
{
"checksumSHA1": "MLGPYuRc7BtUuKmIV8f/jXpYWvg=",
@ -568,26 +567,26 @@
{
"checksumSHA1": "9jjO5GjLa0XF/nfWihF02RoH4qc=",
"path": "golang.org/x/net/context",
"revision": "4971afdc2f162e82d185353533d3cf16188a9f4e",
"revisionTime": "2016-11-15T21:05:04Z"
"revision": "f2499483f923065a842d38eb4c7f1927e6fc6e6d",
"revisionTime": "2017-01-14T04:22:49Z"
},
{
"checksumSHA1": "WHc3uByvGaMcnSoI21fhzYgbOgg=",
"path": "golang.org/x/net/context/ctxhttp",
"revision": "4971afdc2f162e82d185353533d3cf16188a9f4e",
"revisionTime": "2016-11-15T21:05:04Z"
"revision": "f2499483f923065a842d38eb4c7f1927e6fc6e6d",
"revisionTime": "2017-01-14T04:22:49Z"
},
{
"checksumSHA1": "GIGmSrYACByf5JDIP9ByBZksY80=",
"path": "golang.org/x/net/idna",
"revision": "a6577fac2d73be281a500b310739095313165611",
"revisionTime": "2017-03-08T20:54:49Z"
"revision": "f2499483f923065a842d38eb4c7f1927e6fc6e6d",
"revisionTime": "2017-01-14T04:22:49Z"
},
{
"checksumSHA1": "nt4o3cMOyP9prgVeAZgRf82/Et0=",
"checksumSHA1": "TIuteMCFMIoZioMPqSjQYU5O2EY=",
"path": "golang.org/x/net/publicsuffix",
"revision": "0744d001aa8470aaa53df28d32e5ceeb8af9bd70",
"revisionTime": "2017-09-01T09:04:45Z"
"revision": "f2499483f923065a842d38eb4c7f1927e6fc6e6d",
"revisionTime": "2017-01-14T04:22:49Z"
},
{
"checksumSHA1": "HmVJmSDDCwsPJQrp7ml2gXb2szg=",
@ -620,52 +619,52 @@
"revisionTime": "2017-06-22T15:12:08Z"
},
{
"checksumSHA1": "oaglBTpGgEUgk7m92i6nuZbpicE=",
"checksumSHA1": "Mr4ur60bgQJnQFfJY0dGtwWwMPE=",
"path": "golang.org/x/text/encoding",
"revision": "2910a502d2bf9e43193af9d68ca516529614eed3",
"revisionTime": "2016-07-21T22:28:28Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "HgcUFTOQF5jOYtTIj5obR3GVN9A=",
"checksumSHA1": "YhpPlqR+jmjjk330zJXeLGyRlpM=",
"path": "golang.org/x/text/encoding/charmap",
"revision": "3bd178b88a8180be2df394a1fbb81313916f0e7b",
"revisionTime": "2017-07-29T23:24:55Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "zeHyHebIZl1tGuwGllIhjfci+wI=",
"path": "golang.org/x/text/encoding/internal",
"revision": "2910a502d2bf9e43193af9d68ca516529614eed3",
"revisionTime": "2016-07-21T22:28:28Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "TF4hoIqHVEAvOq67rfnSLSkcZ1Y=",
"checksumSHA1": "dteZ0KeAdZy+4iEvL8x+DfZhrnA=",
"path": "golang.org/x/text/encoding/internal/identifier",
"revision": "2910a502d2bf9e43193af9d68ca516529614eed3",
"revisionTime": "2016-07-21T22:28:28Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "G9LfJI9gySazd+MyyC6QbTHx4to=",
"path": "golang.org/x/text/encoding/unicode",
"revision": "2910a502d2bf9e43193af9d68ca516529614eed3",
"revisionTime": "2016-07-21T22:28:28Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "Qk7dljcrEK1BJkAEZguxAbG9dSo=",
"path": "golang.org/x/text/internal/utf8internal",
"revision": "2910a502d2bf9e43193af9d68ca516529614eed3",
"revisionTime": "2016-07-21T22:28:28Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "cQ4+8mXpYioml5/hO7ZyeECoFJc=",
"checksumSHA1": "IV4MN7KGBSocu/5NR3le3sxup4Y=",
"path": "golang.org/x/text/runes",
"revision": "2910a502d2bf9e43193af9d68ca516529614eed3",
"revisionTime": "2016-07-21T22:28:28Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "TZDHZj3zWDc5LKqpoLamOKt6Nmo=",
"checksumSHA1": "ziMb9+ANGRJSSIuxYdRbA+cDRBQ=",
"path": "golang.org/x/text/transform",
"revision": "2910a502d2bf9e43193af9d68ca516529614eed3",
"revisionTime": "2016-07-21T22:28:28Z"
"revision": "11dbc599981ccdf4fb18802a28392a8bcf7a9395",
"revisionTime": "2017-01-12T23:38:57Z"
},
{
"checksumSHA1": "u5J0Ld9kNs8aieexH5dqhFhfqIE=",