Code Browser with Gatsby
Published on December 23, 2023
At first, I was content with the ability to insert code snippets into these articles. gatsby-remark-prismjs makes that process pretty straightforward.
But what if an example is more complex than that? Web developers typically turn to tools like CodeSandbox for this purpose. However, the downside there is that the example code needs to be versioned using e.g. CodeSandbox and cannot be stored alongside our source code.
We'll be taking a fairly deep dive into Gatsby to solve this problem. The general workflow with Gatsby is to use source plugins to add nodes to the object store, then run transformers over them to do useful things (e.g. generating thumbnails or parsing JSON), then finally to query the results in your page components.
First Step: Sourcing and Transforming
There is already a gatsby-source-filesystem
to handle the common case of sourcing arbitrary files, but unless they are transformed we cannot access the file contents directly using this plugin alone.
Enter gatsby-transformer-plaintext
, which does exactly what you might think: transforms sourced files with a MIME type of text/plain
by adding the file's contents into the Gatsby object store.
Unfortunately, this plugin only works for text files and is not configurable. So I forked it into gatsby-transformer-source-code. The new plugin allows you to specify the MIME types you want to process, then puts the file contents in a SourceCode
node attached to each File
node in the Gatsby store.
Now we can configure Gatsby like this:
module.exports = {
plugins: [
{
resolve: 'gatsby-source-filesystem',
options: {
name: 'snippets',
path: `${__dirname}/src/snippets`
}
},
{
resolve: 'gatsby-transformer-source-code',
options: {
mimeTypes: ['application/javascript', 'text/jsx', 'text/x-scss']
}
}
]
};
This lays the foundation for what we will be building. Now the source code files (and their contents) are in Gatsby's object store, waiting for us to query them.
Second Step: Provide Data When Creating Pages
Because of Gatsby's design, we cannot use the useStaticQuery
hook to fetch our code snippet data - it does not support passing/referencing variables. We need to fetch the content dynamically based on URL, so we have to provide some metadata to the page renderer which will then be fed back into a GraphQL query to provide the data at the page component level.
This blog generates a page for an article, each of which is contained in an MDX file. Here is a summarized version of that code from gatsby-node.js
:
exports.createPages = async ({ actions, graphql, reporter }) => {
const mdxComponent = resolve('src/components/mdxArticle.js');
const { createPage } = actions;
const result = await graphql(`
query {
allMdx {
nodes {
internal {
contentFilePath
}
frontmatter {
path
}
}
}
}
`);
result.data.allMdx.nodes.forEach((node) => {
const {
frontmatter: { path },
internal: { contentFilePath }
} = node;
if (!path) {
reporter.warn(
`Did not find a path in the frontmatter of ${contentFilePath}`
);
return;
}
createPage({
// this "URL" is for gatsby-plugin-mdx
component: `${mdxComponent}?__contentFilePath=${contentFilePath}`,
path,
context: {
// this variable will be accessible in the pageQuery
pathGlob: `${path.substring(1)}/**/*`
}
});
});
};
This queries the MDX nodes provided by gatsby-plugin-mdx
and creates a page for each, critically passing pathGlob
along as a page context variable. Without this, we would not be able to filter for "snippets that relate to the page in question."
Third Step: Querying The Data
Now, in the src/components/mdxArticle.js
file, we will use the following page query:
query ($pathGlob: String!) {
allFile(
filter: {
sourceInstanceName: { eq: "snippets" }
relativeDirectory: { glob: $pathGlob }
}
) {
nodes {
path: relativePath
code: childSourceCode {
... on SourceCode {
content
}
}
}
}
}
Each article will now load any snippets that live under a path matching the article URL. For example, if an article has the URL /react/some-article-name
, then ./src/snippets/react/some-article-name
will be searched for code.
Now we have the data at the page level. However, this is an MDX page - the actual markdown content is passed as the children
prop to the page component. Because of this, we will have to leverage a React context to span the gap.
Fourth Step: Integrating With MDX
import CodeBrowser from 'src/components/codeBrowser';
export default function MdxArticle({ data, children }) {
const {
mdx: { frontmatter },
allFile: { nodes: snippets }
} = data;
return (
<ArticleContainer {...frontmatter}>
<MDXProvider components={{ CodeBrowser }}>
<SnippetProvider
snippets={snippets.map((snippet) => ({
...snippet,
path: snippet.path.replace(`${frontmatter.path.substring(1)}/`, '')
}))}
>
{children}
</SnippetProvider>
</MDXProvider>
</ArticleContainer>
);
}
We use the MDXProvider
that gatsby-plugin-mdx
offers to add in a custom component called <CodeBrowser />
. Then we pass the snippet data we received from GraphQL to the SnippetProvider
, which simply copies them into an otherwise empty context.
import Prism from 'prismjs';
import { uniq } from 'lodash-es';
import PropTypes from 'prop-types';
import { useState, useEffect, useMemo } from 'react';
import { Row, Tab, Col, Card, Container } from 'react-bootstrap';
import DirectoryTree from 'components/directoryTree';
import useSnippets from 'hooks/useSnippets';
const getDirName = (path) => path.substring(0, path.lastIndexOf('/'));
const getExtension = (path) => path.substring(path.lastIndexOf('.') + 1);
export default function CodeBrowser({ id }) {
const { snippets } = useSnippets();
const [activeDocument, setActiveDocument] = useState(null);
useEffect(() => {
const activeDoc = snippets.find(
(doc) => doc.path.replace(`${id}/`, '') === activeDocument
);
if (activeDoc) {
Prism.highlightElement(
document.getElementById(`doc-${activeDoc.path.replace(`${id}/`, '')}`)
);
}
}, [activeDocument]);
const snippetTree = useMemo(() => {
const result = {
path: '.',
files: [],
children: []
};
const strippedSnippets = snippets.map((doc) => ({
...doc,
path: doc.path.replace(`${id}/`, '')
}));
const directories = uniq(
strippedSnippets.map((doc) => getDirName(doc.path))
);
for (const dir of directories) {
const segments = dir.split('/');
if (!segments.join('')) {
continue;
}
for (let i = 0; i < segments.length; i++) {
let node = result;
const path = segments.slice(0, i + 1);
for (const segment of path) {
const nextNode = node.children.find((dir) => dir.path === segment);
if (nextNode) {
node = nextNode;
} else {
node.children.push({
path: segment,
files: [],
children: []
});
}
}
}
}
for (const doc of strippedSnippets) {
let node = result;
const segments = getDirName(doc.path).split('/');
for (const segment of segments) {
const nextNode = node.children.find((dir) => dir.path === segment);
if (nextNode) {
node = nextNode;
}
}
node.files.push(doc);
}
return result;
}, [snippets]);
return (
<Tab.Container
id={id}
activeKey={activeDocument}
onSelect={(path) => setActiveDocument(path)}
>
<Row className="g-2">
<Col xs={3}>
<Card body>
<Card.Title>Files</Card.Title>
<DirectoryTree activeDocument={activeDocument} node={snippetTree} />
</Card>
</Col>
<Col xs={9}>
<Tab.Content>
{snippets.map((doc) => (
<Tab.Pane
eventKey={doc.path.replace(`${id}/`, '')}
key={doc.path}
>
<Container fluid>
<Row className="g-0">
<Col xs={12} className="bg-primary rounded-top">
<h5 className="my-3 ms-3">
{doc.path.replace(`${id}/`, '')}
</h5>
</Col>
</Row>
<Row>
<Col xs={12}>
<pre
id={`doc-${doc.path.replace(`${id}/`, '')}`}
className={`language-${getExtension(
doc.path
)} mt-0 rounded-bottom`}
style={{ maxHeight: 650, overflowY: 'auto' }}
>
{doc.code.content}
</pre>
</Col>
</Row>
</Container>
</Tab.Pane>
))}
</Tab.Content>
</Col>
</Row>
</Tab.Container>
);
}
CodeBrowser.propTypes = {
id: PropTypes.string.isRequired
};
This component renders a directory/file structure on the left with the selected source code on the right.
In our MDX file, we can now simply use the following JSX to embed our code sample.
<CodeBrowser id="example-one" />
Now we can show complete code samples with a file tree browser. Here's the code we just wrote above, showing itself:
gatsby-config.js
module.exports = { plugins: [ { resolve: 'gatsby-source-filesystem', options: { name: 'articles', path: `${__dirname}/articles` } }, { resolve: 'gatsby-source-filesystem', options: { name: 'snippets', path: `${__dirname}/snippets` } }, { resolve: 'gatsby-transformer-source-code', options: { mimeTypes: ['application/javascript'] } } ] };
gatsby-node.js
const { resolve } = require('path'); exports.createPages = async ({ actions, graphql, reporter }) => { const mdxComponent = resolve('src/components/mdxArticle.js'); const { createPage } = actions; const result = await graphql(` { allMdx { nodes { internal { contentFilePath } frontmatter { path } } } } `); if (result.errors) { reporter.panicOnBuild('Error while running GraphQL markdown query.'); return; } let counter = 0; result.data.allMdx.nodes.forEach((node) => { const { frontmatter: { path }, internal: { contentFilePath } } = node; if (!path) { reporter.warn( `Did not find a path in the frontmatter of ${contentFilePath}` ); return; } counter++; createPage({ component: `${mdxComponent}?__contentFilePath=${contentFilePath}`, path, context: { pathGlob: `${path.substring(1)}/**/*` } }); }); reporter.info(`Created ${counter} markdown pages!`); }; exports.onCreateWebpackConfig = ({ actions }) => { actions.setWebpackConfig({ resolve: { modules: [resolve(__dirname, 'src'), 'node_modules'] } }); };
articles/demo.mdx
--- path: /demo date: 2023-12-20 description: Demo title: Demo --- This is an example article. <CodeBrowser id="first" />
src/components/articleContainer.jsx
import Prism from 'prismjs'; import { useEffect } from 'react'; import PropTypes from 'prop-types'; import { format, parseISO } from 'date-fns'; import { Container, Row, Col } from 'react-bootstrap'; import Layout from 'components/layout'; export default function ArticleContainer({ title, description, date, children }) { useEffect(() => { Prism.manual = true; Prism.highlightAll(); }, []); return ( <Layout title={title} description={description}> <Container> <Row> <Col md="12" className="mb-2"> <h1>{title}</h1> {Boolean(date) && ( <h4>Published on {format(parseISO(date), 'yyyy-MM-dd')}</h4> )} </Col> <hr /> <Col xs={12}>{children}</Col> </Row> </Container> </Layout> ); } ArticleContainer.propTypes = { title: PropTypes.string.isRequired, description: PropTypes.string, date: PropTypes.string, children: PropTypes.node.isRequired };
src/components/codeBrowser.jsx
import Prism from 'prismjs'; import { uniq } from 'lodash-es'; import PropTypes from 'prop-types'; import { useState, useEffect, useMemo } from 'react'; import { Row, Tab, Col, Card, Container } from 'react-bootstrap'; import DirectoryTree from './directoryTree.jsx'; import useSnippets from '../hooks/useSnippets'; const getDirName = (path) => path.substring(0, path.lastIndexOf('/')); const getExtension = (path) => path.substring(path.lastIndexOf('.') + 1); export default function CodeBrowser({ id }) { const { snippets } = useSnippets(); const [activeDocument, setActiveDocument] = useState(null); useEffect(() => { const activeDoc = snippets.find( (doc) => doc.path.replace(`${id}/`, '') === activeDocument ); if (activeDoc) { Prism.highlightElement( document.getElementById(`doc-${activeDoc.path.replace(`${id}/`, '')}`) ); } }, [activeDocument]); const snippetTree = useMemo(() => { const result = { path: '.', files: [], children: [] }; const strippedSnippets = snippets.map((doc) => ({ ...doc, path: doc.path.replace(`${id}/`, '') })); const directories = uniq( strippedSnippets.map((doc) => getDirName(doc.path)) ); for (const dir of directories) { const segments = dir.split('/'); if (!segments.join('')) { continue; } for (let i = 0; i < segments.length; i++) { let node = result; const path = segments.slice(0, i + 1); for (const segment of path) { const nextNode = node.children.find((dir) => dir.path === segment); if (nextNode) { node = nextNode; } else { node.children.push({ path: segment, files: [], children: [] }); } } } } for (const doc of strippedSnippets) { let node = result; const segments = getDirName(doc.path).split('/'); for (const segment of segments) { const nextNode = node.children.find((dir) => dir.path === segment); if (nextNode) { node = nextNode; } } node.files.push(doc); } return result; }, [snippets]); return ( <Tab.Container id={id} activeKey={activeDocument} onSelect={(path) => setActiveDocument(path)} > <Row className="g-2"> <Col xs={3}> <Card body> <Card.Title>Files</Card.Title> <DirectoryTree activeDocument={activeDocument} node={snippetTree} /> </Card> </Col> <Col xs={9}> <Tab.Content> {snippets.map((doc) => ( <Tab.Pane eventKey={doc.path.replace(`${id}/`, '')} key={doc.path} > <Container fluid> <Row className="g-0"> <Col xs={12} className="bg-primary rounded-top"> <h5 className="my-3 ms-3"> {doc.path.replace(`${id}/`, '')} </h5> </Col> </Row> <Row> <Col xs={12}> <pre id={`doc-${doc.path.replace(`${id}/`, '')}`} className={`language-${getExtension( doc.path )} mt-0 rounded-bottom`} style={{ maxHeight: 650, overflowY: 'auto' }} > {doc.code.content} </pre> </Col> </Row> </Container> </Tab.Pane> ))} </Tab.Content> </Col> </Row> </Tab.Container> ); } CodeBrowser.propTypes = { id: PropTypes.string.isRequired };
src/components/mdxArticle.jsx
// eslint-disable-next-line no-unused-vars import React from 'react'; import { graphql } from 'gatsby'; import PropTypes from 'prop-types'; import { MDXProvider } from '@mdx-js/react'; import ArticleContainer from './articleContainer.jsx'; import CodeBrowser from './codeBrowser.jsx'; import SnippetProvider from './snippetProvider.jsx'; export default function MdxArticle({ data, children }) { const { mdx: { frontmatter }, allFile: { nodes: snippets } } = data; return ( <ArticleContainer {...frontmatter}> <MDXProvider components={{ CodeBrowser }}> <SnippetProvider snippets={snippets.map((snippet) => ({ ...snippet, path: snippet.path.replace(`${frontmatter.path.substring(1)}/`, '') }))} > {children} </SnippetProvider> </MDXProvider> </ArticleContainer> ); } MdxArticle.propTypes = { data: PropTypes.shape({ mdx: PropTypes.shape({ frontmatter: PropTypes.shape({ path: PropTypes.string.isRequired, title: PropTypes.string.isRequired, description: PropTypes.string, date: PropTypes.string }) }), allFile: PropTypes.shape({ nodes: PropTypes.arrayOf( PropTypes.shape({ path: PropTypes.string.isRequired, code: PropTypes.shape({ content: PropTypes.string.isRequired }) }) ) }) }), children: PropTypes.node.isRequired }; export const pageQuery = graphql` query ($path: String!, $pathGlob: String!) { mdx(frontmatter: { path: { eq: $path } }) { frontmatter { path title description date } } allFile( filter: { sourceInstanceName: { eq: "snippets" } relativeDirectory: { glob: $pathGlob } } ) { nodes { path: relativePath code: childSourceCode { ... on SourceCode { content } } } } } `;
src/components/directoryTree.jsx
import PropTypes from 'prop-types'; import { Fragment, useState } from 'react'; import { Row, Col, Nav } from 'react-bootstrap'; import { FontAwesomeIcon } from '@fortawesome/react-fontawesome'; import { faFolder, faCaretDown, faCaretRight, faFile } from '@fortawesome/free-solid-svg-icons'; const getFileName = (path) => path.substring(path.lastIndexOf('/') + 1); export default function DirectoryTree({ activeDocument, node, level = 0 }) { const { path, children, files } = node; const isFile = !Array.isArray(files); const [expanded, setExpanded] = useState(level === 0); return ( <Fragment> <Row onClick={isFile ? null : () => setExpanded((prevVal) => !prevVal)}> <Col xs={1}> {!isFile && ( <FontAwesomeIcon icon={expanded ? faCaretDown : faCaretRight} /> )} </Col> <Col xs={11} style={level > 0 ? { paddingLeft: (level + 1) * 12 } : {}}> <Nav.Item className={activeDocument === path && 'text-primary'}> <Nav.Link eventKey={isFile ? path : null}> <FontAwesomeIcon icon={isFile ? faFile : faFolder} className="me-1" />{' '} {path ? getFileName(path) : '.'} </Nav.Link> </Nav.Item> </Col> </Row> {expanded && [...(children || []), ...(files || [])].map((child) => ( <DirectoryTree key={child.path} node={child} level={level + 1} activeDocument={activeDocument} /> ))} </Fragment> ); } DirectoryTree.propTypes = { activeDocument: PropTypes.string, node: PropTypes.object.isRequired, level: PropTypes.number };
src/components/snippetProvider.jsx
import PropTypes from 'prop-types'; import { SnippetContext } from '../hooks/useSnippets'; export default function SnippetProvider({ snippets, children }) { return ( <SnippetContext.Provider value={{ snippets }}> {children} </SnippetContext.Provider> ); } SnippetProvider.propTypes = { snippets: PropTypes.arrayOf(PropTypes.object), children: PropTypes.node.isRequired };
src/hooks/useSnippets.js
import { createContext, useContext } from 'react'; export const SnippetContext = createContext({ snippets: [] }); export default function useSnippets() { return useContext(SnippetContext); }
snippets/demo/first/index.js
export default { add: (a, b) => a + b };