Docs/minimization
2
0
Fork 0
Code Issues 19 Pull requests Projects Releases Packages Wiki Activity Actions

init

Browse source
This commit is contained in:
Adam Samalik 2019-06-17 14:28:36 +02:00
parent ad6b5bd6c5
commit 1f7364e792
9 changed files with 360 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

3
.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
build
cache
public

122
README.md
View file

@ -1,3 +1,121 @@
# minimization # Fedora Docs Template
This repository contains a minimal source structure for a new Fedora Docs source.
## Structure
```
|-- README.md
|-- antora.yml ....................... 1.
|-- build.sh ......................... 2.
|-- preview.sh ....................... 3.
|-- site.yml ......................... 4.
`-- modules
`-- ROOT ......................... 5.
|-- assets
| `-- images ............... 6.
| `-- pizza.png
|-- nav.adoc ................. 7.
`-- pages .................... 8.
|-- architecture.adoc
|-- community.adoc
|-- faq.adoc
|-- index.adoc
|-- pizza-dough.adoc
`-- pizza-owen.adoc
```
1. Metadata definition.
2. A script that does a local build. Uses docker.
3. A script that shows a preview of the site in a web browser by running a local web server. Uses docker.
4. A definition file for the build script.
5. A "root module of this documentation component". Please read below for an explanation.
6. **Images** to be used on any page.
7. **Menu definition.** Also defines the hierarchy of all the pages.
8. **Pages with the actual content.** They can be also organised into subdirectories if desired.
## Components and Modules
Antora introduces two new terms:
* **Component** — Simply put, a component is a part of the documentation website with its own menu. Components can also be versioned. In the Fedora Docs, we use separate components for user documentation, the Fedora Poject, Fedora council, Mindshare, FESCO, but also subprojects such as CommOps or Modulartity.
* **Module** — A component can be broken down into multiple modules. Modules still share a single menu on the site, but their sources can be stored in different git repositories, even owned by different groups. The default module is called "ROOT" (that's what is in this example). If you don't want to use multiple modules, only use "ROOT". But to define more modules, simply duplicate the "ROOT" directory and name it anything you want. You can store modules in one or more git repositories.
## Local preview
This repo includes scripts to build and preview the contents of this repository.
**NOTE**: Please note that if you reference pages from other repositoreis, such links will be broken in this local preview as it only builds this repository. If you want to rebuild the whole Fedora Docs site, please see [the Fedora Docs build repository](https://pagure.io/fedora-docs/docs-fp-o/) for instructions.
Both scripts work on Fedora (using Podman) and macOS (using Docker).
To build and preview the site, run:
```
$ ./build.sh && ./preview.sh
```
The result will be available at http://localhost:8080
### Installing Podman on Fedora
Fedora Workstation doesn't come with Podman preinstalled by default — so you might need to install it using the following command:
```
$ sudo dnf install podman
```
### Preview as a part of the whole Fedora Docs site
You can also build the whole Fedora Docs site locally to see your changes in the whole context.
This is especially useful for checking if your `xref` links work properly.
To do this, you need to clone the main [Fedora Docs build repository](https://pagure.io/fedora-docs/docs-fp-o), modify the `site.yml` file to reference a repo with your changes, and build it.
Steps:
Clone the main repository and cd into it:
```
$ git clone https://pagure.io/fedora-docs/docs-fp-o.git
$ cd docs-fp-o
```
Find a reference to the repository you're changing in the `site.yml` file, and change it so it points to your change.
So for example, if I made a modification to the Modularity docs, I would find:
```
...
- url: https://pagure.io/fedora-docs/modularity.git
branches:
- master
...
```
And replaced it with a pointer to my fork:
```
...
- url: https://pagure.io/forks/asamalik/fedora-docs/modularity.git
branches:
- master
...
```
I could also point to a local repository, using `HEAD` as a branch to preview the what's changed without the need of making a commit.
**Note:** I would need to move the repository under the `docs-fp-o` directory, because the builder won't see anything above.
So I would need to create a `repositories` directory in `docs-fp-o` and copy my repository into it.
```
...
- url: ./repositories/modularity
branches:
- HEAD
...
```
To build the whole site, I would run the following in the `docs-fp-o` directory.
```
$ ./build.sh && ./preview.sh
```
Fedora Minimization Objective

16
antora.yml Normal file
View file

@ -0,0 +1,16 @@
# Name will be mostly visible in the URL. Treat it as an indentifier.
# Tip: If you want to use the local preview scripts that come with this repository, please change this value in the site.yml file as well. (under site/start_page)
name: minimization # <---- PLEASE MODIFY
# Title will be visible on the page.
title: Minimization # <---- PLEASE MODIFY
# If you don't plan to have multiple versions of the docs (for example, to document multiple versions of some software), you can ignore this field. Otherwise, change "master" to a specific version.
version: master
# We encourage you to name the index page as "index.adoc". If you absolutely have to use a different name, please reflect it here. You can ignore this field otherwise.
start_page: ROOT:index
# This lists all the menu definitions of your component.
nav:
- modules/ROOT/nav.adoc

46
build.sh Executable file
View file

@ -0,0 +1,46 @@
#!/bin/sh
image="docker.io/antora/antora"
cmd="--html-url-extension-style=indexify site.yml"
if [ "$(uname)" == "Darwin" ]; then
# Running on macOS.
# Let's assume that the user has the Docker CE installed
# which doesn't require a root password.
echo ""
echo "This build script is using Docker container runtime to run the build in an isolated environment."
echo ""
docker run --rm -it -v $(pwd):/antora $image $cmd
elif [ "$(expr substr $(uname -s) 1 5)" == "Linux" ]; then
# Running on Linux.
# Check whether podman is available, else faill back to docker
# which requires root.
if [ -f /usr/bin/podman ]; then
echo ""
echo "This build script is using Podman to run the build in an isolated environment."
echo ""
podman run --rm -it -v $(pwd):/antora:z $image $cmd
elif [ -f /usr/bin/docker ]; then
echo ""
echo "This build script is using Docker to run the build in an isolated environment."
echo ""
if groups | grep -wq "docker"; then
docker run --rm -it -v $(pwd):/antora:z $image $cmd
else
echo ""
echo "This build script is using $runtime to run the build in an isolated environment. You might be asked for your password."
echo "You can avoid this by adding your user to the 'docker' group, but be aware of the security implications. See https://docs.docker.com/install/linux/linux-postinstall/."
echo ""
sudo docker run --rm -it -v $(pwd):/antora:z $image $cmd
fi
else
echo ""
echo "Error: Container runtime haven't been found on your system. Fix it by:"
echo "$ sudo dnf install podman"
exit 1
fi
fi

1
modules/ROOT/nav.adoc Normal file
View file

@ -0,0 +1 @@
* xref:index.adoc[Fedora Minimization Objective]

121
modules/ROOT/pages/index.adoc Normal file
View file

@ -0,0 +1,121 @@
= Fedora Minimization Objective
Objective lead: Adam Samalik (asamalik)
== Vision
Thousands of individual and corporate contributors collaborate in the Fedora community to explore new problems and to build a fast-moving modern OS with a rich ecosystem allowing them to experiment on modernising their infrastructure.
The Fedora community produces the number one Linux OS for modern deployments such as containers and IoT. Fedora enables users to build small container and other images with the desired functionality by providing optimised dependency trees of its rich ecosystem, and an ability to fine-tune various installations to achieve the right balance between features and size.
== The problem
While Fedora is well suited for traditional physical/virtual workstations and servers, it is often overlooked for use cases beyond traditional installs.
=== Patching footprint
There is a direct relationship between the installation footprint and attack surface & relevant CVEs. Many users and use cases benefit from reducing the number of patches necessary to keep the system or image updated. Also, reboots are expensive for certain environments and these environments also benefit from a reduced number of patches.
=== Container Image Size
Image size is certainly not the only factor that matters in the container space but it definitely matters especially as the number of images being maintained and regularly respun grows. The Fedora base image has grown to over 300 MB and is roughly three times larger than other mainstream distributions like Ubuntu, Debian, & openSUSE which range from 91-113 MB.
=== Package Dependencies
RPMs in Fedora often use hard dependencies for cases where that's not necessary — making the resulting installation bigger.
== Goals
**Problem exploration**: Explore the problem of dependency trees becoming too big over time and minimize them. The ultimate goal is to make the size of things built on top of our images (containers, ISOs, etc.) smaller.
**Packaging optimization**: Optimize dependencies of selected use cases to minimize their installation footprints, making the maintenance of their production deployments simpler (less things to worry about), and also their development faster (less things for CI to test).
**Data and feedback for smaller images**: Provide feedback backed by data to other teams responsible for creating images (containers, ISOs, etc.) to help them maintain the right balance between the image size and including useful content.
**Mindshare**: Write blog posts, present at conferences, and use other communication channels to show that Fedora cares about minimal but usable installations.
== Outcomes
**Fedora could become even more popular**: Thanks to targeting the emerging use cases such as containers and IoT, and communicating that through different channels, Fedora could become even more popular for these use cases.
**Patching footprint minimized**: Running Fedora in production will mean having fewer dependencies present on a running system.
**Small yet useful container base and application images**: Optimized dependency trees and data generated by this objective will help define and build smaller images that still contain useful components that are generally expected to be there.
**Better packaging experience**: Tooling and services, and potential infrastructure improvements will help packagers to make the right decisions regarding dependencies with less effort.
**Faster Fedora CI**: Focusing on the minimized installations of the most popular use cases optimized by this objective the Fedora CI will be able to target those specific use cases and test them faster because there will be less packages in total.
== Strategy
=== Technical Strategy
**Step zero**: focus on the most popular use cases: We will focus on the most popular use cases instead of the distribution in general. Defining these use-cases is part of this objective. We will then optimize those use cases. As a bonus, the area where they overlap will very roughly define a good starting point for content for our images that we potentially look into as a part of one of our stretch goals.
**Step one: packages and their dependencies**: Inspect various dependency trees of the most popular use-cases. Because multiple packages can potentially provide a certain dependency, an exploration of multiple installations will be necessary. There are multiple ways how to achieve different results — pre-installing certain packages on the target system, providing additional parameters to libsolv regarding weights of individual packages or the criteria of the overall transaction in terms of size and other aspects. This will be explored more deeply.
**Step two: files and packaging optimization**: Optimize the most viable dependency trees by looking at relations on the filesystem level, searching for ways how to potentially use weak dependencies, `%doc` macros, and other mechanisms such as splitting packages into smaller units in order to make the final installation smaller, with features being installable as optional.
**Usefulness over size**: Regarding images, we believe they should be useful first, and minimal second. Producing tiny images without the expected functionality would not be helpful. However, delivering the expected functionality in a minimal image is definitely beneficial.
**Images are not our direct goal**: We will not create smaller base images directly as a part of this very objective. Instead, we use the data we collect when optimizing the apps and runtimes we focus on and make suggestions backed by data to the people maintaining the images. But we might build some preview images on the side for demonstration purposes.
**Using RPM**: We're doing this with RPM, using features such as `--nodocs`, potentially creating alternative module streams with slimmed-down versions if that makes sense, etc. We're not achieving minimization by deleting files after installation. This might be obvious, but still worth mentioning.
=== Execution Strategy
**Scripts over manual labour**: Write scripts that will perform dependency analysis rather than doing the analysis manually. Even though this means more work short-term, the ability to inspect new versions of content as it appears (and potentially use this script in CI) is more beneficial long-term.
**Large changes on the side first**: In case there is a substantial change required to be made to the infrastructure or elsewhere, we deploy our own infrastructure using other resources rather than testing in the production environment. We will also do large experiments on the side. We believe that staging is not for experiments, it is to validate production-ready deployments before actually going in to production. We need a space to fail fast. After we test and validate, we will present a change proposal to the community.
**Building an environment first**: Our primary output is vision, tooling, services, and people being on board. We explore and demonstrate what is possible, socialise our vision and strategy, and develop tooling and services that help packagers make the right decisions. The team will use those tools to achieve results, of course, but enabling members of the community to help with this objective at their own pace is the primary goal.
**Mentoring and consulting**: With whatever we come up with, we offer active help to Fedora packagers to make sure they're able to get the full benefit of our tooling and services, etc. We will then write testimonials together explaining what packagers have achieved and how.
**Do not reinvent the wheel**: This feels very obvious, but from experience keeping this in mind helps us to actively search for existing resources before creating something new.
== Deliverables & Timeline
=== Phase 1: Discovery (summer)
Preliminary discovery and analysis of the most popular use cases (a very basic one has already started on [github asamalik/container-randomness](https://github.com/asamalik/container-randomness) producing a [report](https://asamalik.fedorapeople.org/container-randomness/report.html).
Published blog posts (and maybe videos) about the process, about what is possible, and about our intentions. This will likely result in feedback and engagement from the community.
Talk at Flock 2019 about the objective, featuring some of the discoveries and plans for what's next.
Specific plans for the next phase.
=== Phase 2: Experiments (fall)
Blog posts and conference talks about what works and what doesn't.
A set of use cases with optimized dependencies (potentially built on the side) available.
Initial versions of tooling and services, and a set of guiding principles for packagers to help them make the right decisions.
Specific plans for the next phase.
=== Phase 3: Stabilization (winter)
Out of the experiments from the previous phase, we will choose the ones to focus on and formulate the plan.
Improved tooling and services that are usable by packagers.
Even more blog posts and conference talks (DevConf.cz and FOSDEM?).
Specific plans for the next phase.
=== Phase 4: Integration (spring/f32)
A set of requirements for the Fedora Infra and RelEng teams to implement changes that have been proven working and needed.
Potential changes in the Packaging Guidelines applied.
Tooling and services available in Fedora.
The dependency chains of the selected apps and runtimes in Fedora should become smaller at this point.
Active help to packagers in form of mentorship, workshops, and other means.
Testimonials that we write together with packagers that will demonstrate what they have achieved and how.

14
nginx.conf Normal file
View file

@ -0,0 +1,14 @@
server {
listen 80;
server_name localhost;
location / {
root /antora/public;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}

18
preview.sh Executable file
View file

@ -0,0 +1,18 @@
#!/bin/sh
if [ "$(uname)" == "Darwin" ]; then
# Running on macOS.
# Let's assume that the user has the Docker CE installed
# which doesn't require a root password.
echo "The preview will be available at http://localhost:8080/"
docker run --rm -v $(pwd):/antora:ro -v $(pwd)/nginx.conf:/etc/nginx/conf.d/default.conf:ro -p 8080:80 nginx
elif [ "$(expr substr $(uname -s) 1 5)" == "Linux" ]; then
# Running on Linux.
# Fedora Workstation has python3 installed as a default, so using that
echo ""
echo "The preview is available at http://localhost:8080"
echo ""
cd ./public
python3 -m http.server 8080
fi

21
site.yml Normal file
View file

@ -0,0 +1,21 @@
site:
title: Local Preview
start_page: minimization::index
content:
sources:
- url: .
branches: HEAD
ui:
bundle:
url: https://asamalik.fedorapeople.org/ui-bundle.zip
snapshot: true
default_layout: with_menu
#supplemental_files: ./supplemental-ui
output:
clean: true
dir: ./public
destinations:
- provider: archive
runtime:
pull: true
cache_dir: ./cache