awk-скрипт для преобразования простого языка разметки в troff или html

Я создал простой язык разметки на основе Unicode под названием начинается и awk(1) скрипт для преобразования текста, написанного на Incipit, в HTML или troff -ms.

Ниже приведен пример текста в incipit, результат которого, преобразованный в troff -ms, а затем в ps и pdf, будет здесь

                                Incipit:
                 A Unicode-based Text Markup Language.
                        Seninha (aka phillbush).

The ‘Incipit Markup Language’ (or ‘Incipit’, for short) is a plain text
markup language that uses Unicode characters and the structure of the
text itself to format documents.  ‘Incipit’ is also the name of an
‘awk(1)’ script that converts ‘Incipit’ documents to html or troff.

In the ‘Incipit Markup Language’, a paragraph is a block of text
delimited by blank lines.  A paragraph may be preceded by a section
header and succeded by a figure.  Enumerations (also knonw as “lists”)
are special types of paragraphs (although I interpret a enumeration as
the continuation of the paragraph before it).

In this document, I call “period” sentences delimited by a period.
I also call “colon” a segment of text delimited by a colon or
semi-colon.

§ Punctuation

Incipit uses Unicode characters (called “punctuation” in this document),
alongside the structure of the text, to format documents.  For example,
the section character (`§`, U+00A7) is used to markup section headers.
The bullet character (`•`, U+2022) is used to markup bulleted lists.

Punctuations are mostly used within a paragraph.  Those punctuations
(called inline punctuation) markup emphases, references, topics or
preformated text.  Punctuations cannot be nested: a portion of text
is either emphasized, or it is preformated, never both.  The types of
inline punctuations are enumerated below.
• Emphasis: Text ‘between single quotes’ is emphasized.  It is formated
            in italic font and the punctuation is removed in the final
            document.
• Topic: Text “between double quotes” is topicalized.  It is formated
         in italic font and the punctuation is kept in the final
         document.
• Reference: Text «between double angle quotes» is reference.  More on
             that in a paragraph below.
• Preformated: Text `between grave accents` or {between curly braces}
               is preformated.  It is formated in monospaced font and
               the punctuation is removed in the final document.
• Meta text: Text ⟨between angle braces⟩ is meta text.  It is formated
             in monospaced font and the punctuation is kept in the final
             document.

.Typing punctuation.
If you use Unix, you can either configure your keybindings or configure
the Compose key to insert punctuation and other characters not found on
a regular keyboard.


§§ Sections

Paragraphs can be grouped in sections, which can be nested.  A section
is a line whose first characters are section punctuations (`§`, U+00A7).
A section can be marked by a one or more section punctuations.  The
number of time that punctuation occurs represents the level of the
section.  For example, a first-level section begins with `§`; a
second-level section begins with `§§`, and so on.


§§ Enumerations

An enumeration, also known as list, is a hierarchical grouping of
periods, called the enumeration items.  Each item begins with zero
or more tab characters followed the enumeration punctuation (`•`,
U+2022), also knon as ‘bullet’.  The number of tabs in the beginning
of an item identifies the item level: zero tab for first-level items;
one tab for second-level items; and so on.

.Enumeration label.
When formated, each enumeration item is usually preceded by a bullet.
However, it can be changed by following the enumeration punctuation by
a string between parentheses.  This can be used for ordered lists, when
the label is a number or letter.

.Enumeration incipit.
Each enumeration item can have a incipit colon, which will be explained
on the “§ Incipit” section below.  The incipit colon is a colon
describing the topic of the item.

The following is an example of enumeration.
• (A) First item:
      This is the first item of a labeled enumeration.
      This item also contains an incipit colon.
• (B) Second item:
      This is the second item of a labeled enumeration.
      It also contains an incipit colon.
• (C) Third item.
    • First subitem of third item.
    • Second subitem of third item.
    • Third subitem of third item.
    • Fourth subitem of third item.
• (D) Fourth item.
• (E) Fifth item.


§ Incipit

The word “incipit” comes from the Latin and means “it begins”.  The
incipit of a text is the first few words of the text.  In the ‘Incipit
Markup Language’, incipits are initial elements of the text used to
format the text itself.  The incipit of a document is its first
paragraph (which contains the title and some meta information); the
incipit of a paragraph is its first period (aka sentence); the incipit
of a period is its first colon (the part separated by colon).

In the ‘Incipit’ Markup Language, a text unit can have no incipit.
A document without incipit is a document without title.
A paragraph without incipit is a paragraph without its special first period.
This implies that certain units of text are made up of two parts:
an optional incipit and a body.

.The incipit of a document.
The first paragraph of a document is its incipit.  If the document
begins with a blank line or with a figure or enumeration, the document
has no incipit.  The first period (ie', the first sentence) of the
document's incipit is the title.  If this period has a incipit colon,
this colon is the main title and the rest is the subtitle.  For example,
this document has an incipit paragraph, which has an incipit period (the
full title), which has an incipit colon (the main title).  The remaining
periods are interpreted depending on the output format.  In troff, the
second period is the author, the third period is the institution, and
the following periods are the abstract of the document.

.The incipit of a paragraph.
If the first period of a paragraph begins with a period punctuation,
this paragraph contains a incipit period.  This incipit period, called
the title of the paragraph, is formated in bold font.  In the source
plain text of this document, the incipit of a paragraph is written alone
in aline above the rest of the paragraph; but this is not necessary, the
incipit period can be written in the same line of the rest of the
paragraph.

.The incipit of a period.
In an enumeration, the first colon of the first period of a enumerated
item is the item's incipit colon.  This incipit colon, called title of
the enumeration, is formated in bold font.  The first enumeration of
this document, listing the types of inline punctuations, contains
incipit colons.


§ Figures

Figures are text delimited between curly brackets.  The opening curly
bracket must be the last character in a line and the closing curly
bracket must be the first character in a line.

The most simple example of figure is code listing, an example of which,
copied from the second edition of “The C Programming Language” book, is
presented below.  Code is usually idented with a tab, so the first tab
of each code line is removed in the final document.

{
    #include <stdio.h>

    main()
    {
        printf("hello, worldn";
    }
}


§§ Images

TODO.


§§ Tables

TODO.


§§ Quotation

TODO.


§ Script

There is an ‘awk(1)’ script (actually, a shell script calling ‘awk(1)’)
used for converting text from the ‘Incipit Markup Language’ into troff
(using the -ms macro) or html.  The manual of this command is presented
below.

{
    INCIPIT(1)           General Commands Manual          INCIPIT(1)

    NAME
           incipit - format incipit document

    SYNOPSIS
           incipit [-T format] file...

    DESCRIPTION
           incipit convert files marked up in the Incipit Markup
           Language into troff using the -ms format (the default) or
           into html.  The formated file is written to standard
           output.
    
           The options are as follows:
    
           -T format
                  Convert to format (either "ms" or "html").  If not
                  supplied, consider "ms" as default.
    
    SEE ALSO
           troff(1), ms(7)
}

Вот сценарий awk:

#!/bin/sh

# show usage and exit
usage() {
    echo "usage: incipit [-T output] [file]" >&2
    exit 1
}

while getopts 'T:' ch
do
    case "$ch" in
    T)
        type="$OPTARG"
        ;;
    *)
        usage
        ;;
    esac
done
shift $(( OPTIND - 1 ))

awk -v "type=$type" '
# unget token
function ungettok() {
    if (!eof) {
        savedtok = 1
    }
}

# get token
function gettok(    a, n) {
    if (eof)
        return -1
    if (savedtok) {
        savedtok = 0
        return 1
    }
    if (length(line) == 0) {
        if ((getline line) <= 0) {
            eof = 1
            return -1
        } else {
            n = split(line, a)
            if (n == 0) {
                toktype = BLANK
                tok = line
                line = ""
            } else if (match(line, /^t*• *(([^)]+))? *([^:]+:)?/)) {
                toktype = ENUMMARK
                tok = substr(line, RSTART, RLENGTH)
                line = substr(line, RSTART + RLENGTH)
            } else if (match(line, /^(§)+[ t]*/)) {
                toktype = SECTIONMARK
                tok = substr(line, RSTART, RLENGTH)
                sub(/[ t]*/, "", tok)
                line = substr(line, RSTART + RLENGTH)
            } else if (line ~ /[ t]*{$/) {
                toktype = FIGUREMARK
                tok = line
                line = ""
            } else {
                toktype = NONE
                tok = line
                line = ""
            }
        }
    } else {
        toktype = NONE
        tok = line
        line = ""
    }
    return 1
}

# get id from string
function genid(s) {
    gsub(/(§)[ t]*/, "#", s)
    gsub(/[ t]+/, "-", s)
    gsub(/[^-#A-Za-z0-9_/]/, "", s)
    s = tolower(s)
    return s
}

# print code (for code figures)
function printcode(s) {
    sub(/^t/, "", s)
    if (troff) {
        gsub(/\/, "\e", s)
        gsub(/"/, "\(dq", s)
        gsub(/'"'"'/, "\(aq", s)
        gsub(/`/, "\(aa", s)
        gsub(/-/, "\-", s)
        sub(/^./, "\\\&&", s)
    }
    printf "%sn", s
}

# print inline code
function printinlinecode(s) {
    if (troff) {
        gsub(/\/, "\e", s)
        gsub(/"/, "\(dq", s)
        gsub(/'"'"'/, "\(aq", s)
        gsub(/`/, "\(aa", s)
        gsub(/-/, "\-", s)
    }
    printf "%s", s
}

# print text
function printnormal(s) {
    sub(/^[ t]+/, "", s)
    printf "%sn", s
}

# do inline punctuation expansion text
function expandpunct(after,    before, punct) {
    before = ""
    punct = ""
    while (match(after, /{|}|⟨|⟩|`|‘|’|“|”/)) {
        before = before substr(after, 1, RSTART - 1)
        punct = substr(after, RSTART, RLENGTH)
        after = substr(after, RSTART + RLENGTH)
        if (puncttype) {
            if (puncttype == "{" && punct == "}") {
                before = before markup[type, "PRE", "END"]
                puncttype = ""
            } else if (puncttype == "⟨" && punct == "⟩") {
                before = before markup[type, "META", "END"]
                puncttype = ""
            } else if (puncttype == "`" && punct == "`") {
                before = before markup[type, "PRE", "END"]
                puncttype = ""
            } else if (puncttype == "‘" && punct == "’") {
                before = before markup[type, "EMPHASIS", "END"]
                puncttype = ""
            } else if (puncttype == "“" && punct == "”") {
                before = before markup[type, "TOPIC", "END"]
                puncttype = ""
            }
        } else if (punct == "{" || punct == "`" || punct == "⟨" || punct == "“" || punct == "‘") {
            if (punct == "{") {
                before = before markup[type, "PRE", "BEG"]
            } else if (punct == "`") {
                before = before markup[type, "PRE", "BEG"]
            } else if (punct == "⟨") {
                before = before markup[type, "META", "BEG"]
            } else if (punct == "‘") {
                before = before markup[type, "EMPHASIS", "BEG"]
            } else if (punct == "“") {
                before = before markup[type, "TOPIC", "BEG"]
            }
            puncttype = punct
        }
    }
    return before after
}

# print text marked up with inline punctuation
function printmarkup(s) {
    sub(/^[ t]+/, "", s)
    printf "%sn", expandpunct(s)
}

# parse document title
function title(    id) {
    printf "%s", markup[type, "TITLE", "BEG"]
    printnormal(tok)
    if (substr(tok, length(tok), 1) == ":") {
        if (gettok() > 0) {
            if (toktype == NONE) {
                printf markup[type, "SUBTITLE", "BEG"], genid(tok)
                printmarkup(tok)
                printf markup[type, "SUBTITLE", "END"]
            } else {
                ungettok()
            }
        }
    }
    printf "%s", markup[type, "TITLE", "END"]
}

# parse document incipit (heading)
function heading() {
    printf "%s", markup[type, "HEADING", "BEG"]
    printf "%s", markup[type, "HEADING1", "BEG"]
    printmarkup(tok)
    printf "%s", markup[type, "HEADING1", "END"]
    if (gettok() > 0) {
        if (toktype == NONE) {
            printf "%s", markup[type, "HEADING2", "BEG"]
            printmarkup(tok)
            printf "%s", markup[type, "HEADING2", "END"]
        } else {
            ungettok()
        }
    }
    printf "%s", markup[type, "HEADING", "END"]
}

# parse abstract (part of document incipit)
function abstract() {
    printf "%s", markup[type, "ABSTRACT", "BEG"]
    while (!eof && toktype == NONE) {
        printmarkup(tok)
        if (gettok() > 0) {
            if (toktype == NONE) {
                printf "%s", markup[type, "SUBTITLE", "BEG"]
                printmarkup(tok)
                printf "%s", markup[type, "SUBTITLE", "END"]
            } else {
                ungettok()
            }
        }
    }
    printf "%s", markup[type, "ABSTRACT", "END"]
    ungettok()
}

# parse document incipit
function docincipit(    n) {
    while (!eof && toktype == NONE) {
        if (n == 0) {
            title()
        } else if (n == 1) {
            heading()
        } else if (n == 2) {
            abstract()
        } else {
            ungettok()
            return
        }
        gettok()
        n++
    }
    ungettok()
}

# parse paragraph
function paragraph(    rem) {
    printf "%s", markup[type, "PARAGRAPH", "BEG"]
    if (substr(tok, 1, 1) == ".") {
        rem = ""
        printf "%s", markup[type, "PARATITLE", "BEG"]
        tok = substr(tok, 2)
        match(tok, /^[^.]*[.]? */)
        rem = substr(tok, RSTART + RLENGTH)
        tok = substr(tok, RSTART, RLENGTH)
        printmarkup(tok)
        printf "%s", markup[type, "PARATITLE", "END"]
        if (rem != "")
            printmarkup(rem)
        gettok()
    }
    while (!eof && toktype == NONE) {
        printmarkup(tok)
        gettok()
    }
    printf "%s", markup[type, "PARAGRAPH", "END"]
    ungettok()
}

# parse figure
function figure(    ret, fig) {
    if (tok ~ "^IMAGE:[ t]") {
        fig = IMAGE
        printf "%s", markup[type, "IMAGE", "BEG"]
    } else if (tok ~ "^VIDEO:[ t]") {
        fig = VIDEO
        printf "%s", markup[type, "VIDEO", "BEG"]
    } else {
        fig = CODE
        printf "%s", markup[type, "CODE", "BEG"]
    }
    line = ""
    while ((ret = (getline)) == 1) {
        if ($0 ~ /^}[ t]*$/)
            break
        if (fig == CODE) {
            printcode($0)
        }
    }
    if (ret != 1) {
        eof = 1
    }
    if (fig == CODE) {
        printf "%s", markup[type, "CODE", "END"]
    } else if (fig == IMAGE) {
        printf "%s", markup[type, "IMAGE", "END"]
    } else if (fig == VIDEO) {
        printf "%s", markup[type, "VIDEO", "END"]
    }
}

# parse enumeration
function enumeration(    lvl, enumlvl, label, colon) {
    # TODO: we are not using the markup[] array here, but we should
    # TODO: add support for enumeration in html
    while (!eof && (toktype == NONE || toktype == ENUMMARK)) {
        if (toktype == ENUMMARK) {
            label = ""
            colon = ""
            lvl = 0
            while (tok ~ /^t/) {
                lvl++
                sub(/^t/, "", tok)
            }
            sub(/^• */, "", tok)
            while (enumlvl > lvl) {
                printf "%s", markup[type, "ITEM", "END"]
                printf "%s", markup[type, "ENUM", "END"]
                enumlvl--
            }
            while (enumlvl < lvl) {
                printf "%s", markup[type, "ENUM", "BEG"]
                enumlvl++
            }
            if (match(tok, /^([^)]+)/)) {
                label = substr(tok, RSTART + 1, RLENGTH - 2)
                tok = substr(tok, RSTART + RLENGTH)
                sub(/^ +/, "", tok)
            }
            if (match(tok, /^.+:/)) {
                colon = substr(tok, RSTART, RLENGTH)
            }
            if (troff) {
                if (label != "") {
                    label = label "."
                    printf markup[type, "ITEM", "BEG"], label, 2 + int(length(label) / 2)
                } else {
                    printf markup[type, "ITEM", "BEG"], "\(bu", 2
                }
            } else {
                printf "%s", markup[type, "ITEM", "BEG"]
            }
            if (colon != "") {
                printf "%s", markup[type, "COLON", "BEG"]
                printnormal(colon)
                printf "%s", markup[type, "COLON", "END"]
            }
        } else {
            printmarkup(tok)
        }
        gettok()
    }
    ungettok()
    while (enumlvl > 0) {
        printf "%s", markup[type, "ITEM", "END"]
        printf "%s", markup[type, "ENUM", "END"]
        enumlvl--
    }
}

# parse section
function section(    lvl) {
    lvl = (type == "html") ? 1 : 0
    while (tok ~ /^(§)/) {
        lvl++
        sub(/^(§)/, "", tok)
    }
    if (gettok() < 0)
        return
    printf markup[type, "SECTION", "BEG"], lvl, genid(tok)
    printmarkup(tok)
    printf markup[type, "SECTION", "END"], lvl
}

# parse the entire document
function document() {
    printf "%s", markup[type, "HEADER"]
    while (gettok() > 0) {
        if (toktype == SECTIONMARK) {
            section()
        } else if (toktype == ENUMMARK) {
            enumeration()
        } else if (toktype == FIGUREMARK) {
            figure()
        } else if (toktype == NONE && NR == 1) {
            docincipit()
        } else if (toktype == NONE) {
            paragraph()
        }
    }
    printf "%s", markup[type, "FOOTER"]
}

# set constants and call the parser
BEGIN {
    if (type == "")
        type = "ms"

    troff = (type == "ms" || type == "man")
    context = ""
    puncttype = ""
    tok = ""
    eof = 0

    NONE = 0

    # token type
    BLANK = 1
    SECTIONMARK = 2
    ENUMMARK = 3
    FIGUREMARK = 4

    # figure types
    CODE = 1
    IMAGE = 2
    VIDEO = 3

    # ms markup
    markup["ms", "HEADER"] = ".de VSn.DSn.nfn.ft CWn..n.de VEn.ft Pn.fin.DEn..n"
    markup["ms", "FOOTER"] = ""
    markup["ms", "TITLE", "BEG"] = ".TLn"
    markup["ms", "TITLE", "END"] = ""
    markup["ms", "SUBTITLE", "BEG"] = ".brn"
    markup["ms", "SUBTITLE", "END"] = ""
    markup["ms", "HEADING", "BEG"] = ""
    markup["ms", "HEADING1", "BEG"] = ".AUn"
    markup["ms", "HEADING1", "END"] = ""
    markup["ms", "HEADING2", "BEG"] = ".AIn"
    markup["ms", "HEADING2", "END"] = ""
    markup["ms", "HEADING", "END"] = ""
    markup["ms", "ABSTRACT", "BEG"] = ".AB non"
    markup["ms", "ABSTRACT", "END"] = ".AEn"
    markup["ms", "PARAGRAPH", "BEG"] = ".PPn"
    markup["ms", "PARAGRAPH", "END"] = ""
    markup["ms", "PARATITLE", "BEG"] = ".Bn"
    markup["ms", "PARATITLE", "END"] = ".Rn"
    markup["ms", "SECTION", "BEG"] = ".NH %dn"
    markup["ms", "SECTION", "END"] = ""
    markup["ms", "CODE", "BEG"] = ".VSn"
    markup["ms", "CODE", "END"] = ".VEn"
    markup["ms", "EMPHASIS", "BEG"] = "\fI"
    markup["ms", "EMPHASIS", "END"] = "\fP"
    markup["ms", "TOPIC", "BEG"] = "\(lq\fI"
    markup["ms", "TOPIC", "END"] = "\fP\(rq"
    markup["ms", "PRE", "BEG"] = "\f(CW"
    markup["ms", "PRE", "END"] = "\fP"
    markup["ms", "META", "BEG"] = "\f(CW\[la]"
    markup["ms", "META", "END"] = "\[ra]\fP"
    markup["ms", "COLON", "BEG"] = ".Bn"
    markup["ms", "COLON", "END"] = ".Rn"
    markup["ms", "ENUM", "BEG"] = ".RSn"
    markup["ms", "ENUM", "END"] = ".REn"
    markup["ms", "ITEM", "BEG"] = ".IP \fB%s\fP %dn"
    markup["ms", "ITEM", "END"] = ""

    # html markup
    markup["html", "HEADER"] = ""
    markup["html", "FOOTER"] = "n"
    markup["html", "TITLE", "BEG"] = "<h1 id="%s">"
    markup["html", "TITLE", "END"] = "</h1>"
    markup["html", "SUBTITLE", "BEG"] = "<br/>"
    markup["html", "SUBTITLE", "END"] = ""
    markup["html", "HEADING", "BEG"] = "<p>"
    markup["html", "HEADING1", "BEG"] = "<br/>"
    markup["html", "HEADING1", "END"] = ""
    markup["html", "HEADING2", "BEG"] = "<br/>"
    markup["html", "HEADING2", "END"] = ""
    markup["html", "HEADING", "END"] = "</p>"
    markup["html", "ABSTRACT", "BEG"] = "<p>"
    markup["html", "ABSTRACT", "END"] = "</p>"
    markup["html", "PARAGRAPH", "BEG"] = "<p>"
    markup["html", "PARAGRAPH", "END"] = "</p>"
    markup["html", "PARATITLE", "BEG"] = "<strong>"
    markup["html", "PARATITLE", "END"] = "</strong>"
    markup["html", "SECTION", "BEG"] = "<h%d id="%s">"
    markup["html", "SECTION", "END"] = "</h%d>"
    markup["html", "CODE", "BEG"] = "<figure><pre><code>"
    markup["html", "CODE", "END"] = "</figure></pre></code>"
    markup["html", "EMPHASIS", "BEG"] = "<em>"
    markup["html", "EMPHASIS", "END"] = "</em>"
    markup["html", "TOPIC", "BEG"] = "“<em>"
    markup["html", "TOPIC", "END"] = "</em>”"
    markup["html", "PRE", "BEG"] = "<code>"
    markup["html", "PRE", "END"] = "</code>"
    markup["html", "META", "BEG"] = "<code>⟨"
    markup["html", "META", "END"] = "⟩</code>"
    markup["html", "COLON", "BEG"] = "<strong>"
    markup["html", "COLON", "END"] = "</strong>"
    markup["html", "ENUM", "BEG"] = "<ul>"
    markup["html", "ENUM", "END"] = "</ul>"
    markup["html", "ITEM", "BEG"] = "<li>"
    markup["html", "ITEM", "END"] = "</li>"

    document()
}
' "$@"