.. | ||
node_modules/is-plain-obj | ||
types | ||
index.js | ||
license | ||
package.json | ||
readme.md |
unified is an interface for processing text using syntax trees. It’s what powers remark (Markdown), retext (natural language), and rehype (HTML), and allows for processing between formats.
Intro
unified enables new exciting projects like Gatsby to pull in Markdown, MDX to embed JSX, and Prettier to format it. It’s used in about 500k projects on GitHub and has about 25m downloads each month on npm: you’re probably using it. Some notable users are Node.js, Vercel, Netlify, GitHub, Mozilla, WordPress, Adobe, Facebook, Google, and many more.
- To read about what we are up to, follow us Twitter
- For a less technical and more practical introduction to unified, visit
unifiedjs.com
and peruse its Learn section - Browse awesome unified to find out more about the ecosystem
- Questions? Get help on Discussions!
- Check out Contribute below to find out how to help out, or become a backer or sponsor on OpenCollective
Sponsors
Support this effort and give back by sponsoring on OpenCollective!
Gatsby 🥇 |
Vercel 🥇 |
Netlify![]() |
Holloway |
ThemeIsle |
Boost Hub![]() |
Expo |
|||
You? |
Install
npm:
npm install unified
This package comes with types.
If you’re using TypeScript, make sure to also install
@types/unist
.
Use
var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')
var report = require('vfile-reporter')
unified()
.use(markdown)
.use(remark2rehype)
.use(doc, {title: '👋🌍'})
.use(format)
.use(html)
.process('# Hello world!', function (err, file) {
console.error(report(err || file))
console.log(String(file))
})
Yields:
no issues found
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>👋🌍</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
Contents
Description
unified is an interface for processing text using syntax trees. Syntax trees are a representation of text understandable to programs. Those programs, called plugins, take these trees and inspect and modify them. To get to the syntax tree from text, there is a parser. To get from that back to text, there is a compiler. This is the process of a processor.
| ........................ process ........................... |
| .......... parse ... | ... run ... | ... stringify ..........|
+--------+ +----------+
Input ->- | Parser | ->- Syntax Tree ->- | Compiler | ->- Output
+--------+ | +----------+
X
|
+--------------+
| Transformers |
+--------------+
Processors
Every processor implements another processor. To create a processor, call another processor. The new processor is configured to work the same as its ancestor. But when the descendant processor is configured in the future it does not affect the ancestral processor.
When processors are exposed from a module (for example, unified
itself) they
should not be configured directly, as that would change their behavior for all
module users.
Those processors are frozen and they should be called to create a
new processor before they are used.
Syntax trees
The syntax trees used in unified are unist nodes.
A node is a plain JavaScript objects with a type
field.
The semantics of nodes and format of syntax trees is defined by other projects.
There are several utilities for working with nodes.
List of processors
The following projects process different syntax tree formats. They parse text to a syntax tree and compile that back to text. These processors can be used as is, or their parser and compiler can be mixed and matched with unified and plugins to process between different syntaxes.
List of plugins
The below plugins work with unified, on all syntax tree formats:
unified-diff
— Ignore messages for unchanged lines in Travisunified-message-control
— Enable, disable, and ignore messages
See remark, rehype, and retext for their lists of plugins.
File
When processing a document, metadata is often gathered about that document. vfile is a virtual file format that stores data, metadata, and messages about files for unified and its plugins.
There are several utilities for working with these files.
Configuration
Processors are configured with plugins or
with the data
method.
Integrations
unified can integrate with the file system with unified-engine
.
CLI apps can be created with unified-args
, Gulp plugins with
unified-engine-gulp
, and Atom Linters with
unified-engine-atom
.
unified-stream
provides a streaming interface.
Programming interface
The API provided by unified allows multiple files to be processed and gives access to metadata (such as lint messages):
var unified = require('unified')
var markdown = require('remark-parse')
var styleGuide = require('remark-preset-lint-markdown-style-guide')
var remark2retext = require('remark-retext')
var english = require('retext-english')
var equality = require('retext-equality')
var remark2rehype = require('remark-rehype')
var html = require('rehype-stringify')
var report = require('vfile-reporter')
unified()
.use(markdown)
.use(styleGuide)
.use(remark2retext, unified().use(english).use(equality))
.use(remark2rehype)
.use(html)
.process('*Emphasis* and _stress_, you guys!', function (err, file) {
console.error(report(err || file))
console.log(String(file))
})
Yields:
1:16-1:24 warning Emphasis should use `*` as a marker emphasis-marker remark-lint
1:30-1:34 warning `guys` may be insensitive, use `people`, `persons`, `folks` instead gals-men retext-equality
⚠ 2 warnings
<p><em>Emphasis</em> and <em>stress</em>, you guys!</p>
Processing between syntaxes
Processors can be combined in two modes.
Bridge mode transforms the syntax tree from one format (origin) to another (destination). Another processor runs on the destination tree. Finally, the original processor continues transforming the origin tree.
Mutate mode also transforms the syntax tree from one format to another. But the original processor continues transforming the destination tree.
In the previous example (“Programming interface”), remark-retext
is used in
bridge mode: the origin syntax tree is kept after retext is
done; whereas remark-rehype
is used in mutate mode: it sets a new syntax
tree and discards the origin tree.
API
processor()
Processor describing how to process text.
Returns
Function
— New unfrozen processor that is configured to work the
same as its ancestor.
When the descendant processor is configured in the future it does not affect the
ancestral processor.
Example
The following example shows how a new processor can be created (from the remark processor) and linked to stdin(4) and stdout(4).
var remark = require('remark')
var concat = require('concat-stream')
process.stdin.pipe(concat(onconcat))
function onconcat(buf) {
var doc = remark().processSync(buf).toString()
process.stdout.write(doc)
}
processor.use(plugin[, options])
Configure the processor to use a plugin and optionally configure that plugin with options.
If the processor is already using this plugin, the previous plugin configuration is changed based on the options that are passed in. The plugin is not added a second time.
Signatures
processor.use(plugin[, options])
processor.use(preset)
processor.use(list)
Parameters
plugin
(Attacher
)options
(*
, optional) — Configuration forplugin
preset
(Object
) — Object with an optionalplugins
(set tolist
), and/or an optionalsettings
objectlist
(Array
) — List of plugins, presets, and pairs (plugin
andoptions
in an array)
Returns
processor
— The processor that use
was called on.
Note
use
cannot be called on frozen processors.
Call the processor first to create a new unfrozen processor.
Example
There are many ways to pass plugins to .use()
.
The below example gives an overview.
var unified = require('unified')
unified()
// Plugin with options:
.use(pluginA, {x: true, y: true})
// Passing the same plugin again merges configuration (to `{x: true, y: false, z: true}`):
.use(pluginA, {y: false, z: true})
// Plugins:
.use([pluginB, pluginC])
// Two plugins, the second with options:
.use([pluginD, [pluginE, {}]])
// Preset with plugins and settings:
.use({plugins: [pluginF, [pluginG, {}]], settings: {position: false}})
// Settings only:
.use({settings: {position: false}})
processor.parse(file)
Parse text to a syntax tree.
Parameters
Returns
Node
— Parsed syntax tree representing file
.
Note
parse
freezes the processor if not already frozen.
parse
performs the parse phase, not the run phase or other
phases.
Example
The below example shows how parse
can be used to create a syntax tree from a
file.
var unified = require('unified')
var markdown = require('remark-parse')
var tree = unified().use(markdown).parse('# Hello world!')
console.log(tree)
Yields:
{
type: 'root',
children: [
{type: 'heading', depth: 1, children: [Array], position: [Position]}
],
position: {
start: {line: 1, column: 1, offset: 0},
end: {line: 1, column: 15, offset: 14}
}
}
processor.Parser
A parser handles the parsing of text to a syntax tree.
Used in the parse phase and called with a string
and
VFile
representation of the text to parse.
Parser
can be a function, in which case it must return a Node
: the
syntax tree representation of the given file.
Parser
can also be a constructor function (a function with a parse
field, or
other fields, in its prototype
), in which case it’s constructed with new
.
Instances must have a parse
method that is called without arguments and must
return a Node
.
processor.stringify(node[, file])
Compile a syntax tree.
Parameters
node
(Node
) — Syntax tree to compilefile
(VFile
, optional) — File, any value accepted byvfile()
Returns
string
or Buffer
(see notes) — Textual representation of the syntax
tree
Note
stringify
freezes the processor if not already frozen.
stringify
performs the stringify phase, not the run phase
or other phases.
unified typically compiles by serializing: most compilers return
string
(or Buffer
).
Some compilers, such as the one configured with rehype-react
,
return other values (in this case, a React tree).
If you’re using a compiler doesn’t serialize, expect different result values.
When using TypeScript, cast the type on your side.
Example
The below example shows how stringify
can be used to serialize a syntax tree.
var unified = require('unified')
var html = require('rehype-stringify')
var h = require('hastscript')
var tree = h('h1', 'Hello world!')
var doc = unified().use(html).stringify(tree)
console.log(doc)
Yields:
<h1>Hello world!</h1>
processor.Compiler
A compiler handles the compiling of a syntax tree to text.
Used in the stringify phase and called with a Node
and VFile
representation of syntax tree to compile.
Compiler
can be a function, in which case it should return a string
: the
textual representation of the syntax tree.
Compiler
can also be a constructor function (a function with a compile
field, or other fields, in its prototype
), in which case it’s constructed with
new
.
Instances must have a compile
method that is called without arguments and
should return a string
.
processor.run(node[, file][, done])
Run transformers on a syntax tree.
Parameters
node
(Node
) — Syntax tree to run onfile
(VFile
, optional) — File, any value accepted byvfile()
done
(Function
, optional) — Callback
Returns
Promise
if done
is not given.
The returned promise is rejected with a fatal error, or resolved with the
transformed syntax tree.
Note
run
freezes the processor if not already frozen.
run
performs the run phase, not other phases.
function done(err[, node, file])
Callback called when transformers are done. Called with either an error or results.
Parameters
err
(Error
, optional) — Fatal errornode
(Node
, optional) — Transformed syntax treefile
(VFile
, optional) — File
Example
The below example shows how run
can be used to transform a syntax tree.
var unified = require('unified')
var references = require('remark-reference-links')
var u = require('unist-builder')
var tree = u('root', [
u('paragraph', [
u('link', {href: 'https://example.com'}, [u('text', 'Example Domain')])
])
])
unified()
.use(references)
.run(tree, function (err, tree) {
if (err) throw err
console.log(tree)
})
Yields:
{
type: 'root',
children: [
{type: 'paragraph', children: [Array]},
{type: 'definition', identifier: '1', title: undefined, url: undefined}
]
}
processor.runSync(node[, file])
Run transformers on a syntax tree.
An error is thrown if asynchronous plugins are configured.
Parameters
node
(Node
) — Syntax tree to run onfile
(VFile
, optional) — File, any value accepted byvfile()
Returns
Node
— Transformed syntax tree.
Note
runSync
freezes the processor if not already frozen.
runSync
performs the run phase, not other phases.
processor.process(file[, done])
Process the given file as configured on the processor.
Parameters
Returns
Promise
if done
is not given.
The returned promise is rejected with a fatal error, or resolved with the
processed file.
The parsed, transformed, and compiled value is exposed on
file.contents
or file.result
(see notes).
Note
process
freezes the processor if not already frozen.
process
performs the parse, run, and stringify phases.
unified typically compiles by serializing: most compilers return
string
(or Buffer
).
Some compilers, such as the one configured with rehype-react
,
return other values (in this case, a React tree).
If you’re using a compiler that serializes, the result is available at
file.contents
.
Otherwise, the result is available at file.result
.
Example
The below example shows how process
can be used to process a file, whether
transformers are asynchronous or not, with promises.
var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')
unified()
.use(markdown)
.use(remark2rehype)
.use(doc, {title: '👋🌍'})
.use(format)
.use(html)
.process('# Hello world!')
.then(
function (file) {
console.log(String(file))
},
function (err) {
console.error(String(err))
}
)
Yields:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>👋🌍</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
function done(err, file)
Callback called when the process is done. Called with a fatal error, if any, and a file.
Parameters
Example
The below example shows how process
can be used to process a file, whether
transformers are asynchronous or not, with a callback.
var unified = require('unified')
var parse = require('remark-parse')
var stringify = require('remark-stringify')
var github = require('remark-github')
var report = require('vfile-reporter')
unified()
.use(parse)
.use(github)
.use(stringify)
.process('@wooorm', function (err, file) {
console.error(report(err || file))
console.log(String(file))
})
Yields:
no issues found
[**@wooorm**](https://github.com/wooorm)
processor.processSync(file|value)
Process the given file as configured on the processor.
An error is thrown if asynchronous plugins are configured.
Parameters
Returns
The parsed, transformed, and compiled value is exposed on
file.contents
or file.result
(see notes).
Note
processSync
freezes the processor if not already frozen.
processSync
performs the parse, run, and stringify
phases.
unified typically compiles by serializing: most compilers return
string
(or Buffer
).
Some compilers, such as the one configured with rehype-react
,
return other values (in this case, a React tree).
If you’re using a compiler that serializes, the result is available at
file.contents
.
Otherwise, the result is available at file.result
.
Example
The below example shows how processSync
can be used to process a file, if all
transformers are synchronous.
var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')
var processor = unified()
.use(markdown)
.use(remark2rehype)
.use(doc, {title: '👋🌍'})
.use(format)
.use(html)
console.log(processor.processSync('# Hello world!').toString())
Yields:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>👋🌍</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
processor.data([key[, value]])
Configure the processor with information available to all plugins. Information is stored in an in-memory key-value store.
Typically, options can be given to a specific plugin, but sometimes it makes sense to have information shared with several plugins. For example, a list of HTML elements that are self-closing, which is needed during all phases of the process.
Signatures
processor = processor.data(key, value)
processor = processor.data(values)
value = processor.data(key)
info = processor.data()
Parameters
key
(string
, optional) — Identifiervalue
(*
, optional) — Value to setvalues
(Object
, optional) — Values to set
Returns
processor
— If setting, the processor thatdata
is called onvalue
(*
) — If getting, the value atkey
info
(Object
) — Without arguments, the key-value store
Note
Setting information cannot occur on frozen processors. Call the processor first to create a new unfrozen processor.
Example
The following example show how to get and set information:
var unified = require('unified')
var processor = unified().data('alpha', 'bravo')
processor.data('alpha') // => 'bravo'
processor.data() // {alpha: 'bravo'}
processor.data({charlie: 'delta'})
processor.data() // {charlie: 'delta'}
processor.freeze()
Freeze a processor. Frozen processors are meant to be extended and not to be configured directly.
Once a processor is frozen it cannot be unfrozen. New processors working the same way can be created by calling the processor.
It’s possible to freeze processors explicitly by calling .freeze()
.
Processors freeze implicitly when .parse()
, .run()
,
.runSync()
, .stringify()
, .process()
,
or .processSync()
are called.
Returns
processor
— The processor that freeze
was called on.
Example
The following example, index.js
, shows how rehype prevents extensions to
itself:
var unified = require('unified')
var parse = require('rehype-parse')
var stringify = require('rehype-stringify')
module.exports = unified().use(parse).use(stringify).freeze()
The below example, a.js
, shows how that processor can be used and configured.
var rehype = require('rehype')
var format = require('rehype-format')
// …
rehype()
.use(format)
// …
The below example, b.js
, shows a similar looking example that operates on the
frozen rehype interface because it does not call rehype
.
If this behavior was allowed it would result in unexpected behavior so an
error is thrown.
This is invalid:
var rehype = require('rehype')
var format = require('rehype-format')
// …
rehype
.use(format)
// …
Yields:
~/node_modules/unified/index.js:440
throw new Error(
^
Error: Cannot invoke `use` on a frozen processor.
Create a new processor first, by invoking it: use `processor()` instead of `processor`.
at assertUnfrozen (~/node_modules/unified/index.js:440:11)
at Function.use (~/node_modules/unified/index.js:172:5)
at Object.<anonymous> (~/b.js:6:4)
Plugin
Plugins configure the processors they are applied on in the following ways:
- They change the processor: such as the parser, the compiler, or configuring data
- They specify how to handle syntax trees and files
Plugins are a concept.
They materialize as attacher
s.
Example
move.js
:
module.exports = move
function move(options) {
var expected = (options || {}).extname
if (!expected) {
throw new Error('Missing `extname` in options')
}
return transformer
function transformer(tree, file) {
if (file.extname && file.extname !== expected) {
file.extname = expected
}
}
}
index.md
:
# Hello, world!
index.js
:
var unified = require('unified')
var parse = require('remark-parse')
var remark2rehype = require('remark-rehype')
var stringify = require('rehype-stringify')
var vfile = require('to-vfile')
var report = require('vfile-reporter')
var move = require('./move')
unified()
.use(parse)
.use(remark2rehype)
.use(move, {extname: '.html'})
.use(stringify)
.process(vfile.readSync('index.md'), function (err, file) {
console.error(report(err || file))
if (file) {
vfile.writeSync(file) // Written to `index.html`.
}
})
Yields:
index.md: no issues found
index.html
:
<h1>Hello, world!</h1>
function attacher([options])
Attachers are materialized plugins. An attacher is a function that can receive options and configures the processor.
Attachers change the processor, such as the parser, the compiler, configuring data, or by specifying how the syntax tree or file are handled.
Context
The context object (this
) is set to the processor the attacher is applied on.
Parameters
options
(*
, optional) — Configuration
Returns
transformer
— Optional.
Note
Attachers are called when the processor is frozen, not when they are applied.
function transformer(node, file[, next])
Transformers handle syntax trees and files.
A transformer is a function that is called each time a syntax tree and file are
passed through the run phase.
If an error occurs (either because it’s thrown, returned, rejected, or passed to
next
), the process stops.
The run phase is handled by trough
, see its documentation for the
exact semantics of these functions.
Parameters
node
(Node
) — Syntax tree to handlefile
(VFile
) — File to handlenext
(Function
, optional)
Returns
void
— If nothing is returned, the next transformer keeps using same tree.Error
— Fatal error to stop the processnode
(Node
) — New syntax tree. If returned, the next transformer is given this new treePromise
— Returned to perform an asynchronous operation. The promise must be resolved (optionally with aNode
) or rejected (optionally with anError
)
function next(err[, tree[, file]])
If the signature of a transformer includes next
(the third
argument), the transformer may perform asynchronous operations, and must
call next()
.
Parameters
err
(Error
, optional) — Fatal error to stop the processnode
(Node
, optional) — New syntax tree. If given, the next transformer is given this new treefile
(VFile
, optional) — New file. If given, the next transformer is given this new file
Preset
Presets are sharable configuration. They can contain plugins and settings.
Example
preset.js
:
exports.settings = {bullet: '*', emphasis: '*', fences: true}
exports.plugins = [
require('remark-preset-lint-recommended'),
require('remark-preset-lint-consistent'),
require('remark-comment-config'),
[require('remark-toc'), {maxDepth: 3, tight: true}],
require('remark-license')
]
readme.md
:
# Hello, world!
_Emphasis_ and **importance**.
## Table of contents
## API
## License
index.js
:
var remark = require('remark')
var vfile = require('to-vfile')
var report = require('vfile-reporter')
var preset = require('./preset')
remark()
.use(preset)
.process(vfile.readSync('readme.md'), function (err, file) {
console.error(report(err || file))
if (file) {
vfile.writeSync(file)
}
})
Yields:
readme.md: no issues found
readme.md
now contains:
# Hello, world!
*Emphasis* and **importance**.
## Table of contents
* [API](#api)
* [License](#license)
## API
## License
[MIT](license) © [Titus Wormer](https://wooorm.com)
Contribute
See contributing.md
in unifiedjs/.github
for ways
to get started.
See support.md
for ways to get help.
Ideas for new plugins and tools can be posted in unifiedjs/ideas
.
A curated list of awesome unified resources can be found in awesome unified.
This project has a code of conduct. By interacting with this repository, organization, or community you agree to abide by its terms.
Acknowledgments
Preliminary work for unified was done in 2014 for
retext and inspired by ware
.
Further incubation happened in remark.
The project was finally externalised in 2015 and published as unified
.
The project was authored by @wooorm.
Although unified
since moved its plugin architecture to trough
,
thanks to @calvinfo,
@ianstormtaylor, and others for their
work on ware
, as it was a huge initial inspiration.