summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorVincent Sanders <vince@kyllikki.org>2017-09-06 10:02:18 (GMT)
committer Vincent Sanders <vince@kyllikki.org>2017-09-06 10:02:18 (GMT)
commit87bb39f155f272047b5c56656cf086eb0b50f9eb (patch)
tree56776a6780fd1478a29226b077984d27fd2620ed
parentdb900faf27516c1985695bcbc4d5be033af7c450 (diff)
downloadnetsurf-wiki-87bb39f155f272047b5c56656cf086eb0b50f9eb.tar.gz
netsurf-wiki-87bb39f155f272047b5c56656cf086eb0b50f9eb.tar.bz2
update for fedora 26
-rw-r--r--continuous_integration.mdwn66
-rw-r--r--continuous_integration/fedora_26_setup.mdwn115
-rw-r--r--virtual_host_server.mdwn35
3 files changed, 175 insertions, 41 deletions
diff --git a/continuous_integration.mdwn b/continuous_integration.mdwn
index 8eeb45d..b3a35b7 100644
--- a/continuous_integration.mdwn
+++ b/continuous_integration.mdwn
@@ -7,34 +7,11 @@
The CI system is a jenkins instance accessed at <http://ci.netsurf-browser.org/>
+
## Master
The master instance is a VPS provided by Mythic beasts.
-A VPN between the master node and all the build slaves provides a
-network where no machines are publicly accessible.
-
-The network uses OpenVPN to create a LAN which other systems
-connect. The [[virtual host server|virtual_host_server]] instances are
-all bridged to the LAN on a single openVPN link. All the physical
-nodes for Mac OS X, armhf etc. are bridged via Vincents router through
-a single openvpn link. The scaleway and other nodes connect with
-separate openvpn connections.
-
-The master node also runs a dnsmasq instance which provides name
-service and DHCP for the CI LAN. The dnsmasq config file has static
-mappings for the CI slaves DHCP responses using 192.168.211.100 for CI
-worker 0 and so on up the range to 192.168.211.130 for worker 30
-
-All virtual machines use locally administered MAC adresses in the
-0E:xx:xx:xx:x:xx prefix range.
-
- - 0e:00:00:00:01:xx used for phoenix VM host
- - 0e:00:00:00:02:xx used for arrch64vmhost VM host
- - 0e:00:00:00:03:xx used for scaleway VM host
-
-In addition ssh based tunnels are used to copy data between CI nodes.
-
The Jenkins install on the master server is a snapshot release version.
### generating openvpn keys
@@ -59,7 +36,7 @@ systems which cannot be virtualised (principally MAC OS X).
### [[Debian 9 (Stretch) Setup|continuous_integration/debian_stretch_setup]]
-### [[Fedora 25 Setup|continuous_integration/fedora_25_setup]]
+### [[Fedora 26 Setup|continuous_integration/fedora_26_setup]]
### [[OpenBSD Setup|continuous_integration/openbsd_setup]]
@@ -71,10 +48,43 @@ systems which cannot be virtualised (principally MAC OS X).
### [[Mac OS X Yosemite Setup|continuous_integration/mac_os_x_yosemite_setup]]
-## Historical
+## Network
-### [[Continuous Integration Debian 8 (Jessie) Setup|continuous_integration/debian_jessie_setup]] - Obsolete
+A VPN between the master node and all the build slaves provides a
+network where no machines are publicly accessible.
-### [[Continuous Integration Debian 7 (Wheezy) Setup|continuous_integration/debian_wheezy_setup]] - Obsolete
+A [[network map|continuous_integration/netsurf_ci_network_diagram.pdf]]
+has been produced to make visualising the private network easier.
+
+The network uses OpenVPN to create a LAN which other systems
+connect. The [[virtual host server|virtual_host_server]] instances are
+all bridged to the LAN on a single openVPN link. All the physical
+nodes for Mac OS X, armhf etc. are bridged via Vincents router through
+a single openvpn link. The scaleway and other nodes connect with
+separate openvpn connections.
+
+The master node also runs a dnsmasq instance which provides name
+service and DHCP for the CI LAN. The dnsmasq config file has static
+mappings for the CI slaves DHCP responses using 192.168.211.100 for CI
+worker 0 and so on up the range to 192.168.211.130 for worker 30
+
+All virtual machines use locally administered MAC adresses in the
+0E:xx:xx:xx:x:xx prefix range.
+
+ - 0e:00:00:00:01:xx used for phoenix VM host
+ - 0e:00:00:00:02:xx used for arrch64vmhost VM host
+ - 0e:00:00:00:03:xx used for scaleway VM host
+
+In addition ssh based tunnels are used to copy data between CI nodes.
+
+## Historical
The old discussion on [[Autobuilder requirements|continuous_integration/autobuilder_requirements]] lists requirements for the new autobuilder.
+
+Obsolete slave setups:
+
+ - [[Debian 8 (Jessie) Setup|continuous_integration/debian_jessie_setup]]
+ - [[Debian 7 (Wheezy) Setup|continuous_integration/debian_wheezy_setup]]
+ - [[Fedora 25 Setup|continuous_integration/fedora_25_setup]]
+
+
diff --git a/continuous_integration/fedora_26_setup.mdwn b/continuous_integration/fedora_26_setup.mdwn
new file mode 100644
index 0000000..27de6e4
--- a/dev/null
+++ b/continuous_integration/fedora_26_setup.mdwn
@@ -0,0 +1,115 @@
+[[!meta title="Continuous Integration worker Fedora 26 Setup"]]
+[[!meta author="Kyllikki"]]
+[[!meta date="2017-09-06T09:44:14Z"]]
+
+[[!toc]]
+
+## Fedora 26 OS install
+
+### amd64 VDS install from media
+
+[[Virtual server setup|virtual_host_server]]
+
+use text mode to install fedora
+
+ 1 Language settings to English/UK
+ 2 Time settings to Europe/London
+ 3 Installation source to local media
+ 4 Software selection to fedora server edition
+ 5 Installation destination to virtual disc, whole disc, standard partition layout
+ 6 Network config to wired (ens3)
+ 7 root password to netsurf ci worker default
+ 8 create netsurf user
+
+Once install is completed virtual server will power off
+
+
+## Manual CI worker install
+
+### required packages
+
+to obtain required packages
+
+ dnf install fedora-packager fedora-review java-1.8.0-openjdk-headless gcc clang git pkgconfig libcurl-devel libjpeg-devel bison flex expat-devel libpng-devel openssl-devel check gperf perl-HTML-Parser ccache
+
+For gtk2 builds
+
+ dnf install gtk2-devel
+
+for gtk3 builds
+
+ dnf install gtk3-devel
+
+Development tool group using
+
+ dnf group install 'Development Tools'
+
+
+### config
+
+On master jenkins use "manage nodes" to create new node. Ensure
+"remote fs root" is set to /var/lib/jenkins add variable JENKINS\_HOME
+set to /var/lib/jenkins
+
+As superuser:
+
+create jenkins user
+
+ adduser --system --create-home --home-dir /var/lib/jenkins/ jenkins
+
+ensure /opt is setup correctly to allow toolchains to be built on the node
+
+ mkdir -p /opt/netsurf
+ chown jenkins:jenkins /opt/netsurf
+
+become jenkins user
+
+ su -s /bin/bash - jenkins
+
+create ssh keypair (accept defaults - no password)
+
+ ssh-keygen -t rsa -C "netsurf@nsciworker17.netsurf-browser.org"
+
+copy .ssh/id\_rsa.pub from worker to jenkins master node and append to /home/netsurf/.ssh/authorized\_keys
+
+ scp .ssh/id_rsa.pub netsurf@ci.netsurf-browser.org:nsciworker17_id_rsa.pub
+
+get jenkins slave jar
+
+ wget http://ci.netsurf-browser.org/jenkins/jnlpJars/slave.jar
+
+exit jenkins user shell
+
+create ns-ci-worker.service file
+
+ [Unit]
+ Description=Netsurf CI worker
+ Documentation=http://wiki.netsurf-browser.org/continuous_integration/
+ Requires=network.target
+ After=multi-user.target
+
+ [Service]
+ Type=simple
+ ExecStart=/bin/java -Djava.awt.headless=true -jar /var/lib/jenkins/slave.jar -jnlpUrl http://ci.netsurf-browser.org/jenkins/computer/ciworker17/slave-agent.jnlp -secret 0123456789abcdef01234567890abcdef
+ Restart=always
+ RestartSec=60
+ StartLimitInterval=0
+ User=jenkins
+
+ [Install]
+ WantedBy=multi-user.target
+
+untill [this bug](https://bugzilla.redhat.com/show_bug.cgi?id=1358476) is resolved it is necessary to symlink the jli library object somewhere the rpath faliure does not affect.
+
+ # ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-1.b12.fc25.x86_64/jre/lib/amd64/jli/libjli.so /usr/lib64/libjli.so
+
+install and start new service
+
+ install -D -m 644 ns-ci-worker.service /usr/lib/systemd/system/ns-ci-worker.service
+ systemctl daemon-reload
+ systemctl start ns-ci-worker
+ systemctl enable ns-ci-worker
+
+
+
+
diff --git a/virtual_host_server.mdwn b/virtual_host_server.mdwn
index 93fcab5..6bcc6c0 100644
--- a/virtual_host_server.mdwn
+++ b/virtual_host_server.mdwn
@@ -21,34 +21,39 @@ If you are creating a new Debian box there is a helper script (new-stretch-vps.s
for example:
- new-stretch-vps.sh ciworker18
+ new-stretch-vps.sh ${VDSNAME}
-will create a system named ciworker18 with 40G of disc and 512MB of memory with the Debian stretch boot iso connected to the cdrom drive.
+will create a system named ciworker18 with 40G of disc and 512MB of
+memory with the Debian stretch boot iso connected to the cdrom drive.
### Fedora
create lvm device for the system
- lvcreate --size 40G -n nsciworker18 vg
+ lvcreate --size 40G -n ${VDSNAME} vg
Ensure the OS install dvd is in /var/lib/libvirt/images/
Create the virtual machine
- virt-install -n nsciworker18 -r 1024 \
+ virt-install -n ${VDSNAME} -r 1024 \
--disk path=/dev/mapper/vg-${VDSNAME},bus=virtio \
--accelerate \
- --network=bridge:${VDSBRIDGE} \
+ --network=bridge:br0 \
--connect=qemu:///system \
--noautoconsole -v \
--nographics \
--os-type=linux \
- --location /var/lib/libvirt/images/Fedora-Server-dvd-x86_64-25-1.3.iso \
+ --location /var/lib/libvirt/images/Fedora-Server-dvd-x86_64-26-1.5.iso \
--extra-args='console=tty0 console=ttyS0,115200n8 serial'
+The bridge interface MAC address should be edited to something like 0e:00:00:00:01:xx
+
+ virsh edit ${VDSNAME}
+
Once running the serial console can be used
- virsh console nsciworker18
+ virsh console ${VDSNAME}
### Other
@@ -57,14 +62,16 @@ The manual setup for other systems is:
create lvm device for the system
- lvcreate --size 40G -n system_name vg
+ lvcreate --size 40G -n ${VDSNAME} vg
-if the OS requires a cdrom install media download and place in /var/lib/libvirt/images/
+if the OS requires a cdrom install media download and place in
+/var/lib/libvirt/images/
-use virt install to create the new VM substitutig appropriate values for volume name, memory, boot disc
+use virt install to create the new VM substitutig appropriate values
+for volume name, memory, boot disc
- virt-install -n ciworker18 -r 512 \
- --disk path=/dev/mapper/vg-ciworker13,bus=virtio \
+ virt-install -n ${VDSNAME} -r 512 \
+ --disk path=/dev/mapper/vg-${VDSNAME},bus=virtio \
-c /var/lib/libvirt/images/cd54.iso \
--accelerate --network=bridge:br0 \
--connect=qemu:///system \
@@ -73,5 +80,7 @@ use virt install to create the new VM substitutig appropriate values for volume
Admin
-----
-You can either use virsh on phoenix to directly admin consoles or alternatively use virt-admin over the VPN which is by far the easier option.
+You can either use virsh on phoenix to directly admin consoles or
+alternatively use virt-admin over the VPN which is by far the easier
+option.