Sunday, June 25, 2017

Asrock E3C224DI-14S IPMI not reachable

Documentation missing

There's definately some documentation missing on the IPMI settings. I managed to lock myself out of the IPMI (also know as 'Integrated Light Out', or ILO) management interface. Not sure how I managed to do that, but in the quest to find out how to restore devine powers, I noticed quite a lot of people suffered from this. And, the solution is quite simple, when you know it. As usual...

Two configurations

The cause of error probably was me updating the network configuration, using the dashboard, instead of using the BIOS update.
IPMI configuration at BIOS
Please note, in the BIOS, you have eth0 and eth1. Eth0 usually is the first interface, so when you assume this would be the IPMI interface, you assumed as I.
Link to Network configuration on the dashboard
Which is completely and utterly wrong... Eth1, which has the label IPMI for a reason, is the correct one, and is found as Port 8 (above the 2 USB ports).

what's with the eth0/eth1, then?

It turns out there's a nifty, quite undocumented feature, of IPMI fallback. the BIOS eth0/NCSI item can be used as IPMI fallback - in case eth1 is not connected... I found this because after the lock out, I could actually use the IPMI when all cables were plugged in. The NCSI port is port 6, or LAN1 (designated as such in the manual), or eth0 (as seen in the BIOS BMC configuration). For completeness sake, LAN2 is not mentioned in the BIOS, only in the Megarac SP configuration (and hard to find).

My recommendation

Stay away from the megarac SP network configration items. Use the BIOS, which takes precedence over the Mearace SP settings anyway, only, and only configure IPMI/eth1 for a fixed ip-adddress. You can always use arp -a to find out the DCHP-assigned ip address to the other interfaces. You can find the MAC-addresses of your LAN1 and LAN2 interfaces at the BMC configuration section of the BIOS, under BMC MAC Restore Tool:
Hope this helps anyone.

Wednesday, June 21, 2017

Stroage server

Storage server


Aiming at 2 VDEVs of 5 or 6 disks each, I'd need a motherboard capable of running 12 disks.I used a SuperMicro board in the ESXi build, mainly because virtualization using bare metal hypervisors was quite new to me. However, these boards have quite a steep price.
There's a new motherboard by SuperMicro, that screams NAS, but that has not yet hit the shops.

So, I ended up with:


It starts off with placing the processor and memory on the motherboard. This is best done outside the case:
One of the reasons I love Fractal cases is the disks cages; not only can you replace the cage with hot plug ones, you may also relocate one or both cages. And the disk frames just slide out - no tools needed.

And, there's room for two SSD's at the back of the motherboard:

The case assemply starts with adding the power supply.
Then, place the motherboard, and attach the power cables. Some cable management is in order, but will be done after all disks have been installed and hooked up.
Time to put these components to the test. Download Memtestx86, or the commercial version, and let it run for a while.
OK, that'll do pig, that'll do.

To be continued with part two of the storage server: Firmware

Storage Server: Firmware


The first thing to do, in order to get any software RAID program to run, is to flash the controller out of RAID mode. Only then all of the disks will be seen as just a bunch of disks - nothing else. JBOD that is, for short.
The board I have, comes with a LSI SAS2308 controller, to with I want to connect 12 SATA drives using three SAS-to-SATA breakout cables.


There are two locations you can get the drivers for those LSI2308 controllers: the SuperMicro site, which at the time of writing offers the P20 version, and the ASRock Rack site, that describes quite nicely how to flash to P16, in order to support freeNAS.

My current verion is P19, RAID mode:
LSI Corporation SAS2 Flash Utility Version (2013.07.19) Copyright (c) 2008-2013 LSI Corporation. All rights reserved Adapter Selected is a LSI SAS: SAS2308_1(D1) Controller Number : 0 Controller : SAS2308_1(D1) PCI Address : 00:02:00:00 SAS Address : 5d05099-0-0000-5198 NVDATA Version (Default) : NVDATA Version (Persistent) : Firmware Product ID : 0x2714 (IR) Firmware Version : NVDATA Vendor : LSI NVDATA Product ID : Undefined BIOS Version : UEFI BSD Version : N/A FCODE Version : N/A Board Name : ASRSAS2308 Board Assembly : N/A Board Tracer Number : N/A
Note the "(IR)" in the Firmware Product ID section; I want that to read "(IT)". And, as freeNAS now supports P20, I'll go for the P20 version, off the SuperMicro site.


Download, unzip and copy the DOS subdirectory contents to a bootable USB stick. I use Rufus and a freedos image for that purpose. Boot from it, and start the flashing:

Never mind the typo :)
After a while, you will see this:

You can find the address as "SAS Address" listed above (which output of the ASRock utility), or you can find it on the controller configuration pages (make sure the controller is marked as bootable, and press CTRL-C)
On this screen copy, you can see the new firmware version, 20, and mode: IT. Which is what I wanted. You can also see the address, by the way, formatted a bit differently. The last nine digit are displayed as '0:00005198'; SuperMicro seems to use a prefix of '5003048', the original ASRock being '5D0599'. We'll find out if it makes a difference in the next sequence: Storage Server: Software!

Monday, June 19, 2017

Things to do after you cloned a Virtual Machine

Clean up a cloned VM

After you made a clone of your (base) VM, you will need to do some stuff.


First of all, I suspect you have a different MAC-address than the original machine. VMWare does that, as long as you have your MAC address assigned automatically. VirtualBox will ask you whether to re-initialize the MAC-address while cloning.
The problem is the udev process, responsable for handling devices. This uses confuguration files, located in/etc/udev/rules.d directory. The file
70-persistent-net.rules will have an entry, based on your original machine. An entry looks like:

# PCI device 0x15ad:0x07b0 (vmxnet3) (custom name provided by external tool) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:71:3c:be", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
You see the MAC-address, as wel as the assigned link (eth0). I already altered this line to reflect the correct MAC-address, and link name. The easiest way turns out to be to remove this file completely; it will be re-generated when absent:
cd /etc/udev/rules.d/ rm -f 70-persistent-net.rules reboot now reboot

eth0 or eth1

Another problem may be the fact your assigned link names went wrong; you may have extra rules in your udev file for eth1. And, your existing eth0 may still carry the wrong MAC-address, which may cause errors like
ifup eth0 eth0 does not seem to be present,delaying initialization.
Check if your MAC-addresses are still lingering around with:
/sbin/ifconfig | grep "^eth" eth0 Link encap:Ethernet HWaddr 00:0C:29:71:3C:BE
This is a correct output - MAC-addresses match. If not, change the /etc/sysconfig/network-scripts/ifcfg-eth0 file to reflect the correct MAC-address:
Depending on the number of NIC's, ifcfg-eth1 may require some tweaking, as well. You should now successfully be able to start networking services. 


Change the hostname; edit /etc/sysconfig/network, and adapt /etc/hosts

Sunday, June 18, 2017

Now, here's an idea...

Gaining control

Or rather - regaining control. Over my own data, and what's done with it.

Currently, I use several services, of which I know they are monitored. Several of these services fall under US legislation, although I'm not a US citizen. This allows several agencies to go through my documents, email and other stuff, whether I like that or not (I do not).

Of course, for some of this, I gave permission - blogging on a google platform undoubly allows google to scan, "in order to enhance services rendered". Or something similar. Using gmail: ditto. Drobox: ditto. MS Windows: Ditto.

And all of these firms store data on US territory, or are US based, which basically tells me my data is being scanned.

Now, I am aware of this, but not overly comfortable with it. I like my privacy. I like the idea of being innocent until proven otherwise.

So, how about taking matters in one's own hands? How about setting up my own email and cloud services?

Services wanted

Just freewheeling here, but how about: 

  • replace gmail by dovecot 
  • replace dropbox by nextcloud (successor/fork of owncloud) 
  • create some virtual/cloud computing platform to replace ESXi. My own Azure, so to speak. 

I do have a previous (not documented) ESXi server build, and I run some 10 virtual machines on it, one of them being FreeNAS - because of it's native ZFS.
This works as a charm, despite the FreeNAS community being.... let's say sceptical, about the idea.
The only problem is that ESXi looses connection, and that is canof hard to re-establish.
So, I felt the urge to build a dedicated, 24/7 storage server. 
This would also have to take care of some laptop storage, and runs 24/7, so it better be energy-efficient. Of course, I'm not able, budget wise, to go for the ultimate option, SSD-only. A mix of SSD and 2.5" drives should do, and I could probably salvage some 3TB disks of the ESXi build. 

Enter Sub-project 1: storage server.

Friday, August 14, 2015

OAM PS3 State-of-the-art

An attempt to run OAM 11G Release 2 PS3 on Oracle Linux 6.7, WLS 12C, RDBMS 12C.

Install Linux

Pretty straightforward. Used Oracle 6.7, as 7 is not certified. Create a 200MB /boot, and an LVM for /, both ext4. Install just the server. Deselect *all* options, just X system and X legacy support (the OUI needs it). Some 566 packages will get installed. Make sure it boots, and the network starts.

Linux Maintenance

Change /etc/sysconfig/selinux to read

I needed to use
ifup eth0
Address that by editing /etc/sysconfig/network-scripts/ifcfg-eth0 and change

Update to the latest:
yum update

As a pleasant surprise, my 6.6 was updated to 6.7, according to the boot messages.
Then, log in as root again, and start preparing for installs:
yum install oracle-rdbms-server-12cR1-preinstall

Also, add the following to /etc/sysctl.conf:
# IPv6 disabled net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1

As the oracle preinstall alters you grub, you do want to... reboot.

Oracle installs

[root@tvs ~]# mkdir /oracle [root@tvs ~]# chown oracle:oinstall /oracle [root@tvs ~]# passwd oracle Changing password for user oracle. New password: Retype new password: passwd: all authentication tokens updated successfully.
As oracle:
[oracle@tvs ~]$ mkdir /oracle/depot
Mount my private storage:
[root@tvs ~]# mount -o soft,intr,rsize=8192,wsize=8192,nolock /oracle/depot


/oracle/depot/12G/database/runInstaller - fails with
"PRVF-0002: could not retrieve local node name".

I have to modify the hosts file (/etc/hosts), and add the current ip address and host name. I chose just to install the software, for a single instance database, changed the base to /oracle/app, and that changes the db software location to /oracle/app/product/12.1.0/dbhome_1. All else remains default.
Alter .bash_profile:
# Additional stuff export ORACLE_HOME=/oracle/app/product/12.1.0/dbhome_1 export PATH=$ORACLE_HOME/bin:$PATH export TNS_ADMIN=$ORACLE_HOME/network/admin


Install the latest Java JDK; WLS 12C needs a development environment...
tar zxf /oracle/depot/weblogic/jdk-8u51-linux-x64.tar.gz -C /oracle mv /oracle/jdk1.8.0_51/* /oracle/java/
Alter .bash_profile, source it, and check:
export JAVA_HOME=/oracle/java/jre export PATH=$JAVA_HOME/bin:$PATH [oracle@tvs] . .bash_profile [oracle@tvs] java -version java version "1.8.0_51" Java(TM) SE Runtime Environment (build 1.8.0_51-b16) Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)
This version is not aligned with the installers, these make Java throw this warning:
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0

Use netca to create a listener, or manually, if you like that:
[oracle@tvs ~]$ cat /oracle/app/product/12.1.0/dbhome_1/network/admin/listener.ora # listener.ora Network Configuration File: /oracle/app/product/12.1.0/dbhome_1/network/admin/listener.ora # Generated by Oracle configuration tools. LISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = tvs)(PORT = 1521)) ) )

If you want to cut-n-paste the above code, make sure to manually start the listener, using lsnrctl start.

Repository Creation Utility


OK, that has no problems installing against a 12C database. I heart (never seen it, though) that previous versions would not install against 12C.

Weblogic 12C

java -jar -d64 /oracle/depot/weblogic/fmw_12.
Do NOT start the Configuration Wizard. Also, make sure, this is the first install in your middleware home - WLS refuses to install in a non-empty location.


Has been done before, nothing new, but for the Java version used here.
/oracle/depot/OFM/ -jreLoc $JAVA_HOME

Just as well create an instance, as described here.


Has been done before, too. Refer to that blog entry.


WLS12C does NOT work with OAM. The never comes to an end other than "Too many parameters". Besides, it's not certified according to the matrix, as is Java 8 (Java 8 is not certified, that is.) So, I guess state of the art is Linux 7 (yes - that is certified!), WLS, Java 7 update 80.

Wednesday, July 08, 2015

OAM PS3 - continued

Allow auto start (production mode) for your scripts:
cd /oracle/user_projects/domains/oam_domain/servers mkdir -p oam_server1/security mkdir -p omsm_server1/security mkdir -p oam_policy_mgr1/security vi oam_server1/security/ cp oam_server1/security/ omsm_server1/security/ cp oam_server1/security/ oam_policy_mgr1/security/
You can now use command line scripts to start the other servers (oam_server1, omsm_server1, and oam_policy_mgr1), like so:
/oracle/user_projects/domains/oam_domain/bin/ oam_server1

Starting using WLS Console (GUI)

In the OAM 11GRel2PS2 setup, I created a "machine". This is not a physical machine, just a weblogic placeholder. I need one to allow the nodemanager to start/stop all servers.
If I don, I will get these messages trying to control (start/stop) a managed server via de admin server console:
So, login on the console, choose to expand the Environment, click the "Lock & Edit' button, and create e New Machine:
Name does not really matter, as said, it is just a place holder, and bears no connection to ant (physical) machine whatsoever. Leave OS type to Other; it is not Unix, and it is not Virtual (it is Linux).
Next, define the nodemanager. By default the nodemanager is created to listen in SSL mode, and it will listen on all addresses, so localhost should do.
In a serious clustered environment, you want to change the nodemanager to listen to the machine name or address, not just localhost - if you would, the nodemanager would not accept calls from the other cluster members.
Now, you can add the Admin, and managed servers to this machine, and control these from the GUI as well. Next task will be protecting resources.