howtos/free_space_in_openshift.md

How to free some space in openshift?

If your builds run into an error looking like:

error: build error: devmapper: Thin Pool has 11561 free data blocks which is 
less than minimum required 11944 free data blocks. Create more free space in 
thin pool or use dm.min_free_space option to change behavior

It is likely that one or several of the nodes has some disk space issues.

Check the disk space on the nodes:

You can do this via ansible by simply running:

ansible -a 'lvs' os_nodes_stg

The output will look somthing like this:

  LV          VG              Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root        GuestVolGroup00 -wi-ao---- <58.59g
  docker-pool vg-docker       twi-a-t---  58.32g             42.00  18.44

os-node02.stg.phx2.fedoraproject.org | CHANGED | rc=0 >>
  LV          VG              Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root        GuestVolGroup00 -wi-ao---- <58.59g
  docker-pool vg-docker       twi-a-t--- <48.60g             32.37  14.81

os-node01.stg.phx2.fedoraproject.org | CHANGED | rc=0 >>
  LV          VG              Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root        GuestVolGroup00 -wi-ao---- <58.59g
  docker-pool vg-docker       twi-a-t---  58.32g             40.75  17.38

os-node04.stg.phx2.fedoraproject.org | CHANGED | rc=0 >>
  LV          VG              Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root        GuestVolGroup00 -wi-ao---- <58.59g
  docker-pool vg-docker       twi-a-t---  58.32g             90.32  28.35

In this case you can see that os-node04.stg has a disk full at 90%.

Free space on a node

There is a cron job running regularly to clean up docker images that aren't needed anymore, but you can also run it manually to free some space.

The command to run is then:

docker rmi $(docker images --filter dangling=true -q)

Source: https://pagure.io/fedora-infrastructure/issue/8643