Daily Newspaper on E-Book Reader via Arch Linux

My friend x23b5 has an Amazon Kindle and wants to read daily newspapers each morning.

Time for a cron job on my Pogoplug to send some newspapers to this Kindle 🙂

There is a pretty cool package called calibre.
It can convert online newspapers to ebook specific files and send it via email to your eBook Reader.

Install calibre

Calibre uses recipes to fetch news and convert those to ebook files like .mobi or .epub.
You can list all available recipes with the following command:

Note: It’s a good idea to create a new email for this task because your password will be saved in plain text. Use a freemailer or create a new user on your own mailserver.
Create the following file in /usr/local/bin/calibre-cron and replace all variables and recipes with yours.
I used another usb stick (cacheStick) to save the temp files because I want to protect my arch linux stick.
To change the output format just change the file extension.

Make it executable:

Execute it and check whether it works.

Now we have to create a cron job to execute this script each morning.
Execute this command before you edit your crontab file.

Edit your crontab file:

> /dev/null 2>&1” because I don’t need any email or log output.

Now enable cron service:

Calibre is pretty memory and cpu intensive and takes pretty long! You should run it overnight otherwise your plug pc will be very slow.

How it looks like:

Kindle newspaper

Setup Samba 4 on Arch Linux

Updated: [03.05.2015] Sync some performance settings with my own personal settings, added a link to some more performance tips (see bottom).
Updated: [28.09.2015] Sync again

Note: Check this article if you want to install arch on your pogoplug.

Samba 4 is out now 🙂
So I’ll make this tutorial for Samba 4 because it seems to be slightly faster.
This article will tell you how to install it on your PogoplugV2 or another PlugPC.
Should be pretty much the same for all arch installations.

Samba 3 is going to be removed once Samba 4 is installed.
Your old config will be saved to /etc/samba/smb.conf.pacorig

Installation of Samba 4 is pretty easy.

Now enable the services.

Create Users

If you want to create shares for multiple users you have to create new Unix user and add this one to samba as well.
To make it clean we will create a group called “samba”.

Now we can add a new user to this group. This user “fabian” is not able to login (-s /sbin/nologin) for security purposes.

To use this user in samba shares you have to add it to samba

Create Shares

We are ready to configure our samba shares.
At the beginning configure
To do so edit /etc/samba/smb.conf

Here is an example configuration.
You have to edit the Share definitions below so it fits your setup.

Optimize performance

The stock performance of samba isn’t that great. Especially with NTFS.
But there are some parameters which will increase Samba performance significantly.

Add all these settings to the global section in your smb.conf file.

The main problem for slow file transfer speeds is NTFS, because NTFS needs much CPU on linux.
Nevertheless there are 2 options which will boost the speed:

write cache size

If this integer parameter is set to non-zero value, Samba will create an in-memory cache for each oplocked file (it does not do this for non-oplocked files). All writes that the client does not request to be flushed directly to disk will be stored in this cache if possible. The cache is flushed onto disk when a write comes in whose offset would not fit into the cache or when the file is closed by the client. Reads for the file are also served from this cache if the data is stored within it.

This cache allows Samba to batch client writes into a more efficient write size for RAID disks (i.e. writes may be tuned to be the RAID stripe size) and can improve performance on systems where the disk subsystem is a bottleneck but there is free memory for userspace programs.

The integer parameter specifies the size of this cache (per oplocked file) in bytes.

Default: write cache size = 0

Example: write cache size = 262144 # for a 256k cache size per file

Some example values are:

(262144 = 256KB – you should test some values it’s pretty memory intensive)

Don’t forget to start the samba services or reboot:

For some additonal performance tips check this blogpost: https://linuxengineering.wordpress.com/2014/08/03/performance-tuning-with-pogoplug-v4/

Installing Arch Linux and setting up everything on Plug Computers like PogoplugV2

Some weeks ago I’ve ordered a so called “Plug Computer”.PogoplugV2
These Computers are very small devices with an ARM SoC.
They are pretty cheap! I’ve got my PogoplugV2 for 10€ 🙂

I was lucky and got the gray version 😀
Some device info:

  • ARMv5te CPU (Marvell Kirkwood 1.2GHz)
  • 256MB RAM
  • 128MB NAND
  • 4 USB 2.0 Ports

Pretty great to use this device as a small homeserver/fileserver. Remember 10€!
I want to use my Pogoplug as a fileserver (for Windows PCs) and DLNA server to stream to my TV.
On top of that maybe some additional stuff in the feature 😉

But the default software sucks. No Samba support, no DLNA, no FTP, …
But this doesn’t matter because there is an Arch Linux for ARM Port.
This is an instruction how to install Arch Linux on a PogoplugV2.
Everything after the Arch Linux installation (Install Webmin, Samba, DLNA, …) can be used for all devices which run Arch Linux for ARM.

Prepare device and install Arch Linux

Arch Linux

Everything we have to do to install Arch Linux are 4 steps.

    1. Enabe SSH Access
    2. Format an USB Stick (min. 2GB size)
    3. Flash a custom Bootloader to boot from your USB Stick
    4. Install Arch Linux

An instruction how to do that is available here: http://archlinuxarm.org/platforms/armv5/pogoplug-v2-pinkgray
There are instuctions for other devices as well.
If you use Windows use PuTTY as a SSH client.

Setup everything

1. Login via SSH

Use PuTTY or another SSH client to access your Pogoplug.

username: root
password: root

 2. Change your root password

First you should change your root password.
Use the following command to do so:

 3. Update Arch Linux

Arch Linux has an own package manager called pacman.
With pacman you can install/remove packages and update Arch Linux.

To update Arch Linux use the following command:

 4. Change hostname and timezone

Note: For GUI lovers: skip this step and set hostname and timezone in the webmin interface (next step)

The default hostname is “alarm”. Let’s change it!

To get all available timezones use:

And to set your timezone:

For example:

Now reboot:

Wait a few seconds and reconnect to SSH.

 5. Install Webmin

Webmin is a pretty cool web-based interface for system administration.
It’s easy to use  and will help you to configure stuff faster.
To install it use:

Now we have to allow access from more IP addresses.
You have to edit the configuration file.

Find the following line: allow=127.0.0.1
Now add a new allow line with your local network broadcast ip or a specific ip and save the file.
For example:

Now we’ll enable & start the webmin service so it autostarts.

To access the Webmin interface open a browser and go to:
https://deviceip:10000

To lower memory usage go to:
Webmin>Webmin Configuration>Advanced Configuration and disable Pre-load Webmin functions library?”

 6. USB Auto Mount

If you want to remove your USB HDD and use it anywhere else it would be cool to have automounting like Windows.
There are a lot of auto mounting mechanisms for Arch Linux but a lot of them are outdated.
This one is using udevil to auto mount all USB HDDs on attach as /media/PARTITION_LABEL. So make sure all partitions have a label!

I want to access my HDDs via Samba to use them in Windows so I have to use either FAT32 or NTFS.
This sucks because FAT32 isn’t able to handle files >4GB so it’s useless.
And NTFS is fucking slow on Linux.

But with some special mount options we are able to increase the speed dramatically!
I’ve did some benchmarks with hdparm and dd before and after the optimization.
The read speed was pretty good already (about 29MB/s –> USB 2.0 limit).
But the write speed was really bad!

Before: ~6,5 MB/s write speed
After: ~28,5 MB/s write speed

As you can see the write speed on NTFS was really slow before. But there is a way to fix it 🙂
We have to edit the mount options and add a special ntfs-3g option to our udevil automount settings.
Open the udevil config file and edit it:

Search for default_options_ntfs= and allowed_options=

Now we add the option “big_writes” to both lines so it looks like:

Save the file.

Create the /media directory

Add the udevil service to autostart.

Let’s reboot to see whether it works

Wait a few seconds and reconnect to SSH.
Now you’ll have a new folder: /media/yourHDDname where your HDD is mounted.
And you should see something like /dev/sdX1 on /media/… if you type:

Note: All USB HDDs should go to standby automatically.
You can check with:

7. Blink LED to HDD activity

The PogoplugV2 has a green and orange LED.
So I thought it would be a cool idea change the color to HDD activity.
I’ve created an systemd service.

———————————

Only If you have a PogoplugV2!

You have to correct your arcNumber and matchid because there is a bug in the current uboot and your Pogoplug isn’t detected as a PogoplugV2.

———————————

We need the iostat binary to create our own deamon.
Iostat will check hdd activity.

Create a new file called /usr/lib/systemd/system/blinkled.service with the following content:

Create another file called /usr/local/bin/blinkled
This command is pretty long 😉 It will execute iostat every 3 seconds. If there is HDD activity the Pogoplug will start to blink orange.
You can change heartbeat to default-on or timer as well.

And make it executable

To start our service use:

To autostart use:

 8. Update Kernel to >3.2

Warning: This specific instruction works for PogoplugV2 only!
You can brick your device!

The current kernel for PogoplugV2 is 3.1.x but 3.8.x or newer is available.
You have to update manually because newer kernels need a new uboot.

You need the newest uboot to boot Kernels >3.2 so you have to install it before:
If this is a fresh installation you probably have the newest uboot already and you don’t have to do that!

Then you have to correct your arcNumber and matchid if you didn’t follow “Blink LED to HDD activity”.

Now you can install the new Kernel files with:

 9. Install Samba

See this post

10. Install MiniDLNA

Install MiniDLNA and enable the service.

Now edit the config file

It’s pretty easy to understand the config file and the documentation is well.
You have to set at least one media_dir
All USB devices are mounted at /media/DEVICE_LABEL
For example:

Here are some additional options you could set:

MiniDLNA will update the DB automatically.
If you have a lot of files (>8192) you should increase your max inotify value for a single user so MiniDLNA can watch all files.
Create /etc/sysctl.d/90-inotify.conf and insert the following:

MiniDLNA is able to use thumbnails or cover images.
I’ve created a small bash script which will generate thumbnails for videos recursively.
You need ffmpegthumbnailer to create these thumbnails.

Let’s create the script:

This script will generate a thumbnails for every video file recursively.
You can add even more extensions to the TYPES array.

See: https://github.com/Obihoernchen/generateThumbs

Make it executable:

Execute it with something like:

Now start MiniDLNA

12. Configure static IP

Be careful you can mess up your whole network connection!

I don’t like dynamic IP addresses in my LAN. Especially for servers.
They should have a static IP like every server. Of course there are hostnames but not all devices are able to resolve them.

Arch uses netcfg to configure your network adapters. You can create multiple profile.
To create a new static ip profile create a new profile in /etc/network.d/ named wired-static-eth0

And add the following content, replace everything with your values and save the file.
Note: I use custom DNS settings because my router doesn’t allow me to edit DNS settings.

Now you need to change the default network profile to the new one.

Find NETWORKS= and edit it:

You have to be sure that everything is correct otherwise you are going to loose network connection and have to edit all files from your USB stick with another PC.
Just restart the netcfg service or restart to apply the new settings

 13. Enable color in Bash prompt

The current terminal doesn’t look that nice. Everything is white and boring…
But we can change this so your terminal will look like this:

Terminal with color

Just replace /etc/bash.bashrc with the following content:

More information and other examples: https://wiki.archlinux.org/index.php/Color_Bash_Prompt

To search for available packages in repos if your command doesn’t exist on your current system configuration install pkgfile

14. Disable some logging to extend USB stick lifetime

Flash drives have limited number of write cycles and default linux logging will write pretty much so your USB will be destroyed pretty fast.
That’s why a swap file on your USB stick is a really bad idea!
But you can disable some logging to extend the liefetime of your USB stick.

Just edit syslog-ng.conf and disable some of the “log { source(src); filter…” lines at the end of the file.
You can comment out (add a “#” in front of the line) all lines if you want to disable everything.

I’ve disabled some of them. That’s my example:

On top of that you could disable logging of some servers like samba, minidlna and so on.
I don’t do that because I want to have these logs but to do so set the log path in the config files to /dev/null