Thursday, January 29, 2015

pcDuino3 Nano - Debian / Open Media Vault

This is another in the series documenting my setup of a new home server with the Linksprite pcDuino3 Nano.  A listing for the entire series can be found here.  More information on the pcDuino3 Nano can be found at Linksprite’s website and the pcDuino website.

Back in December when I started my pcDuino3 Nano Home Server Project I outlined how I was replacing my current home server which uses Open Media Vault running on a Raspberry Pi.  In my pcDuino3 Nano – NAS or ??? post I talked about deciding to use something else as OMV neither OMV nor Debian the flavor of Linux that it runs on has a large installed base on the pcDuino.

As I moved forward with the project and started pulling together the software that I wanted to use for my home server I decided that while a home server could certainly be built from scratch the integration that you get with OMV was worth seeing if I could get it up and running on the pcDuino3 Nano.

Thanks to the excellent build scripts from Igor Pecovnik I have had some initial success and now have OMV running on the pcDuino3 Nano.  I plan to document the build and install process in future posts but I wanted to pass along what I have working in case anyone else is looking to run Debian on the pcDuino3 Nano.

Here are some links:

If you have any feedback please post it in the comments section below.  I will attempt to address any questions as quickly as possible.

Saturday, January 10, 2015

Quick Note - Windows 8.1 Corrupt OS Disk - Take 2

Here is another take on accessing the data on the corrupt Windows 8.1 BitLocker secured drive.

SystemResuceCd to boot the system from.

Rufus to create a bootable USB flash drive from an ISO file.

ddrescue to create a image of the deivce.  I used an external USB drive so that when I was done with the image I could easily move the drive with the image to a working system.

Vhdtool to convert the image created by ddrescue to VHD format so it can be mounted under Windows.  You need to rename the image after running the program to .vhd.  Source on github.

Windows Disk Management to mount the VHD file as a drive.

You are prompted for the BitLocker password key.

Drive may require fixing.  Mine did and it took a long time.

Thanks to Andy at cobaltfish.com for the posting that I created my process from.

It Worked for Me - WinPE, BitLocker and a Corrupt Windows 8.1 OS Disk

Recently my daughter’s Window 8.1 laptop developed problems with it’s SSD drive.  Before I could get a good backup of the data on the system it stopped booting.  I had setup this computer as a rush job earlier this year and hadn’t any time to play with it or another Window 8.1 computer.

The first challenged I faced was that the SSD drive was encrypted with BitLocker.  Fortunately, I was able to get the recovery keys for the system and then was able using WinPE to build a USB flash drive that allowed me to boot the system, unlock the drive and backup the data to another USB flash drive.

Below are the steps / notes I took while doing this.  I am posting them so that I have them for the next time and in the hopes that they might help someone.

The first step was to install WinPE and build a bootable USB flash drive.  I had problems getting WinPE for Windows 8.1 to install on my Windows 7 desktop.  Fortunately, there was still a copy of WinPE for Windows 8.0 that I was able to get installed.

WinPE Bootable USB Flash Drive

Install Windows ADK. Then open command window as administrator and run the batch file to define the environment variables.  I installed using the defaults.

After searching the web and reading several sets of instructions I followed the steps below to create the bootable flash drive. Installed all cabs below but believe that only the ones in purple were needed.

The copype.cmd command creates a WinPE environment in the directory specified.  I was using 64bit machines so I used amd64.  For 32 bit you would use x86.

The dism commands mount the boot image file and allow you to manipulate it.  I installed several additional packages using the /add-package feature.  You will need to install these packages to manage-bde which is used to unlock the drive.

C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Deployment Tools\DandISetEnv.bat

copype.cmd amd64 c:\winpe_amd64

dism /mount-wim /wimfile:c:\winpe_amd64\media\sources\boot.wim /index:1 /mountdir:c:\winpe_amd64\mount

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-WMI.cab"

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-FMAPI.cab"

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-SecureStartup.cab"

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-EnhancedStorage.cab"

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\en-us\WinPE-WMI_en-us.cab"

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-Scripting.cab"

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\en-us\WinPE-Scripting_en-us.cab"

dism /image:C:\winpe_amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\8.0\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\en-us\WinPE-SecureStartup_en-us.cab"

On other thing that I did was to create text file key.txt in the WinPE root directory with the BitLocker recovery key in it.  This allowed me using notepad once WinPE had booted to copy and paste the recovery key.  While this isn’t required it sure saves typing.

Then I dismounted the boot image file committing my changes with the /commit option and used the MakeWinPEMedia command to copy my files to the usb flash drive e:.

dism /unmount-Wim /mountdir:c:\winpe_amd64\mount /commit

MakeWinPEMedia /UFD C:\WinPE_amd64 e:

Once the WinPE USB flash drive was created I put it in the laptop and booted it.  The laptop was seutp to boot using UEFI and secure boot mode both of which had to be turned off in the BIOS so that I could select boot from USB mode.

Once I did this the system booted right up. 

Unlock Bitlocker with manage-bde

Once the system was booted I unlocked the BitLocker OS drive using the command below.

manage-bde -unlock c: -recoverypassword <recovery key>

Copy Files off the System

Once the drive was unlocked I used robocopy to copy the files to another USB flash drive.

robocopy c:\<dir> e:\<dir> /copyall /e /r:0 /dcopy:t

Given the number of files it took a good bit of time but at the end I had all the files on the USB flash drive and could load them back on the system after replacing the drive.

Saturday, January 3, 2015

Olimex MOD-WIFI-ESP8266-DEV - Software Setup

Getting to the point where I had a workable software setup for the ESP8266 took me a bit.

ESP8266 Software Setup – Linux

I made use of CNLohr’s excellent instructions and youtube video and the Toolchain wiki page on the ESP8266 Community Forum GitHub repository.

The setup that I ended up with is running on Lubuntu 14.04 with the ESP8266 software under /opt/Espressif based on the ESP8266 Community Forum wiki instructions.

ESP8266 Sample Programs

I have successfully gotten the AT v20 Demo to compile and load and the blinky source code example from the ESP8266 Community Forum following the instructions on the wiki.

For the blinky example you need to connect an LED to GPIO2 which is pin 20 on the MOD-WIFI-ESP8266-DEV.  This was the first program I got running correctly.  I got the basic_example to compile and download but wasn’t ever able to get any output from it.

I had some problems getting the AT v20 Demo to compile.  When I was finished with the wiki instructions I had an include directory in /opt/Espressif and also one in /opt/Espressif/ESP8266_SDK.  The ESP8266 Community Forum template includes /opt/Espressif/include as EXTRA_INCDIR.  However to get the AT V20 Demo to compile I had to combine /opt/Espressif/include into /opt/Espressif/ESP8266_SDK/include.

I also had some problems realizing it was compiling and flashing correctly as I didn’t realize that it was looking for the command line to end with CR LF until I read a post in the ESP8266 Community Forums about needing to press CTRL-J.

ESP8266 Software Setup – Windows (Updated Jan 4, 2015)

For windows I found the very well put together ESP8266 DevKit by CHERTS on the ESP8266 Community Forum.  It comes with an installer and for me with two minor changes “worked right out of the box.”  I was already running CYGWIN64 and am not a bit Eclipse user so I just downloaded the x64 DevKit for windows.

I have to update the COM port used for make flash from COM2 to COM3 and the path to the esptool.py script needed the slashed flipped to work in CYVGWIN.  From the examples directory I ran the following:

$ find . -name '*Makefile' -exec sed -i -e 's/COM2/COM3/g' {} \; 
$ find . -name '*Makefile' -exec sed -i -e 's/c:\\Espressif\\utils\\esptool.exe/c:\/Espressif\/utils\/esptool.exe/g' {} \;

One other thing that I did was to put a shell script in /usr/local/bin for screen.  CYGWIN has the screen command but there doesn’t appear to be a way to change RTS and DTR which creates problems for the ESP8266.  The shell script runs Realterm with DTR and RTS set to 0 which sets the control lines to 3.3v which allows normal operation of the ESP8266.

/usr/local/bin/screen

#!/bin/sh
"c:/Program Files (x86)/BEL/Realterm/realterm.exe" BAUD=9600 RTS=0 DTR=0 PORT=3

My thanks to CHERTS for making this so easy.

Olimex MOD-WIFI-ESP8266-DEV - Hardware Setup

Over the past couple of days I have taken a break from my home server upgrade and have been working with the Olimex MOD-WIFI-ESP8266-DEV board.  It took me a bit but I have come up with the setup below which is working very well for me.

Hardware Setup

The MOD-WIFI-ESP8266-DEV is plugged into the left end of a 400 tie point solderless breadboard.  In the middle of the breadboard is a modified Modern Devices USB BUB I.  On the right side of the breadboard is a v3.3 K$931 low voltage regulator that is plugged into a 5v wall wart power supply.

image

The connections are as listed below.

ESP8266-DEV Connection
1 – 3.3v 3.3v
2 – GND GND
3 – GPIO1 / U0TXD RX1 on the USB BUB I
4 – GPIO3 / U0RXD TX1 on the USB BUB I
13 – RSTB Pull to GND to reset.

RTS on the USB BUB I

To one leg of a momentary switch.  The other leg of the momentary switch is connected to GND.  This provides a reset button.
19 – GPIO15 GND / 3.3vJumper (currently unused)
20 – GPIO2 GND / 3.3vJumper (currently unused)
21 – GPIO0 / MTD0 Pull to GND on boot to enter UART flash upload mode.

DTR on the USB BUB I

This setup allows esptool.py to toggle DTR and RTS to reboot the ESP8266 and put it in UART flash upload mode. 

Serial Terminal Emulator Setup Notes

If you have DTR and RTS from your “serial port” connected to the ESP8266 you need to be careful when you are using a terminal emulation program for output from the ESP8266.  You need to make sure that the terminal emulator doesn’t set RTS low and DTR low for normal operation or the ESP8266 will always boot into UART flash mode. 

  • For linux I am using gtkTerm which has the ability to toggle RTS and DTR.
  • For windows I am using Realterm which allows you to set RTS and DTR

One other note about terminal emulators, for the AT demo programs version 19 and 20 the ESP8266 is looking for a CR and then LF (aka CTRL-M and then a CTRL-J) as a the command line terminator.  This means that with most terminal emulators which generally only send a CR (or only an LF) when you press the enter key you will not get an OK back when you type in “AT ENTER”.  This can be overcome by typing “AT CTRL-M CTRL-J”.

Tuesday, December 30, 2014

pcDuino3 Nano - Postfix Install and Setup via Webmin

This is another in the series documenting my setup of a new home server with the Linksprite pcDuino3 Nano.  A listing for the entire series can be found here.  More information on the pcDuino3 Nano can be found at Linksprite’s website and the pcDuino website.

The next step in setting up my home server was to setup email so that I could relay messages from the server through gmail for monitoring purposes.

This is part of my pcDuino3 Nano series of posts but it should work for webmin on other systems.

Install Postfix from Webmin

From the Webmin menu select Un-used Modules (1), scroll down and select Postfix Mail Server (2).  Then on the resulting page select Click Here to install (3).

image

This will install Postfix.

WebminPostfixInstall

Go back to the Webmin menu and select Servers (1), Postfix Mail Server (2) and then SMTP Authentication And Encryption (3).

image

Enter “[smtp.gmail.com]:587” in the Send outgoing mail box (1), then enter your user name (2) and password (3) and press Save and Apply (4).

image

Back on the Postfix Mail Server page select Virtual Domains (1).

image

Then add a mapping file by selecting the radio button (1) next to Map specification and then select “…” to the right of the box (2) and it will bring up another window.  In this window add a mapping file (3).  I used /etc/postfix/generic.  Then select Save (4) in that window and Save and Apply (5) in the original window.

image

This will take you back to the Postfix Mail Server page where you will again select Virtual Domain.  This time you want to add a new mapping (1).

image

Add a description for the mapping (1), the local username (2) and the username it maps to (3).  Then select Save Mapping (4).  You will want to setup a mapping for any user on the server that will be sending mail.  For alerting purposes root should be enough.

image

When you are done adding mappings select Save and Apply (1) to exit back to the main Postfix Mail Server page.

image

Now you want to select Edit Config Files (1).

image

You will be adding two sections to the main.cf file (1).  Then above the smtp_sasl_password_maps line add “smtp_sasl_auth_enable=yes” (2).  Then scroll to the bottom of the file.

image

At the bottom of the file add the lines below and make sure that there is a newline at the end (3).  Then select Save (4).

##### client TLS parameters #####
smtp_tls_loglevel=1
smtp_tls_security_level=encrypt
smtp_sasl_security_options = noanonymous
smtp_generic_maps = hash:/etc/postfix/generic

image

Now everything should be setup to work.  Testing is most easily done from a command prompt.  ssh to the server and log into root.  If you haven’t already you will need to install mailutils.  Then use the command below to send a test message.

root@testshares:~# apt-get install mailutils
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  mailutils-mh
The following NEW packages will be installed:
  mailutils
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 0 B/344 kB of archives.
After this operation, 910 kB of additional disk space will be used.
Selecting previously unselected package mailutils.
(Reading database ... 89273 files and directories currently installed.)

<messages removed to save space>

update-alternatives: using /usr/bin/mail.mailutils to provide /usr/bin/mailx (mailx) in auto mode.
root@testshares:~# echo "Test Message" | mail -s 'Test 1' user1@gmail.com

This message should end up in user1@gmail.com's mailbox. If it doesn’t you can use the tail command to see the log to see what happened.

root@testshares:~# tail /var/log/mail.log

Thanks to Steve Stonebraker at http://brakertech.com/configure-postfix-with-gmail-via-webmin/ for documenting these instructions.

Monday, December 29, 2014

pcDuino3 Nano - Installing, Setting Up and Testing Samba via Webmin

This is another in the series documenting my setup of a new home server with the Linksprite pcDuino3 Nano.  A listing for the entire series can be found here.  More information on the pcDuino3 Nano can be found at Linksprite’s website and the pcDuino website.

The first step in setting up my home server was to install and configure Samba.

These steps start with installing Samba using Webmin and configure the network for Windows.  From there they create a user, group, add the user to Samba, create a share and set protections on the share so that anyone in the group can access and create files on the share.

I am by no means a Samba expert.  I ran through the steps several times but that doesn’t mean that there could not be problems with them.  If something doesn’t work for you as shown below leave a comment and I will try and figure out what is wrong.

The final configuration for the home server will have the shares on a separate partition from the root file system.  These basic instructions create a testshare on the root drive.  I will build on them in a later post for the final setup.

This is part of my pcDuino3 Nano series of posts but it should work for webmin on other systems.

Install SAMBA from Webmin

From the Webmin menu select Un-unsed Modules (1) and scroll down and select Samba Windows File Sharing (2)

image

You will get a message that the samba server executable was not found and will be asked if you want to install samba. Click on the link to install samba.

image

Select “Click Here” to install and samba will be installed.

image

Now under Servers you will have a Samba Windows File Sharing selection.

image

Configure Network

From the Webmin menu open Networking (1) and select Network Configuration (2) and then select Hostname and DNS Client (3).

image

Here you can setup a name for your server (1), set your DNS Servef (2), change the resolution order to DNS (3) and the Hosts file and press Save (4).

image

After save on the next web page select Apply Configuratio (1)

image

Now go back under Servers to Samba Windows File Sharing and select Windows Networking.

image

Enter your workgroup and server name.

image

Create a User

From the Webmin menu open System (1), select Users and Groups (2) and select Create a new user (3).

image

Enter a username (1), the user’s real name (2), set the shell as preferred (3), set a password (4), create or put the user in an existing group (5), add secondary groups of users and ssh (6) and then press Create (7)

image

You should now see your user in the user list.

image

Add the User to Samba

Go back to Servers (1), select Samba Windows File Sharing (2) and under Samba Users select Convert Users (3).

image

I would suggest changing the default settings to Yes  for deleting unix users that no longer exist and to lock accounts for newly created users.

image

After you select Create Users your users will be created.

image

From back on the share list screen under Samba Users if you select Samba Users (1) you should see the user you created.

image

Select that user (1).

image

Update the users password (1) and select Save (2).

image

Creating a File Share

Go back to Servers (1), select Samba Windows File Sharing (2) and under at the top of the page select Create a new file share (3).

image

Enter a Share name (1), the directory that will be chard (2), the create permissions (3) and the group (4).  For this share I plan to use a group and put all the users who should have access to these files in that group.  As a result I setup the permissions on the directory to be 770 so group and owner have complete access but other has none.  Select Create (5) to create the share.

image

Your new share will show up in the share list at the top of the Samba Windows File Sharing page.  Select the share (1).

image

First you want to select Security and Access Control (2).

image

On the Edit Security page you will need to set Limit to possible list (1) and then add the group under Possible Groups (2) and Read / Write Groups (3).  Then select Save (4).

image

This will return you to the Edit File Share page where you now want to select File Permissions (2).  On the Edit File Permissions page you want to set New Unix file (1) and directory mode (2) to 770 and force the unix group to your group in this case testgroup (3).  Then force Unit file (4) and directory mode (5) to 0770 and 770 respectively and select Save (6).  This will result in files being owned by the user who created them and by the testgroup.  Also, other will have no access to files on the share.

image

Go back to the main Samba Windows File Sharing page and Restart the Samba Server.

image

You now should have a file share that you can map to from the user you created with the password you set in Samba which is different that the logon password.  When you connect to the share using that username and password you should be able to save files to it.