How to support old OSX version with a recent xcode.

About

In this post i decided to share my experience about supporting build for old MacOSX versions using recent xcode/clang. I found that this topic is very tricky and documentation is not covering many parts of it.

How things should work

I am currently using OSX 10.14.1 as a build host with a latest Command Line Tools installed (by xcode-select --install). SDK 10.14.1 should support compilation for any OSX starting from 10.9 to the 10.14. We can specify version to use using -mmacosx-version-min= switch which is used by compiler/linker to generate correct binaries from the target system. Sounds brilliant! So lets test this and see why it is not always as good as it could be.

Testing with plain C code

Lets start with a simple code. I will use clock_gettime as an example. This POSIX function was added in OSX 10.12 and frequently used in opensource software. Here is a sample code i will use:

/*
 simple gettime test
 */
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>
#define BILLION  1000000000L;

int main( int argc, char **argv )
  {
    struct timespec start, stop;
    double accum;

    if( clock_gettime( CLOCK_REALTIME, &start) == -1 ) {
      perror( "clock gettime" );
      exit( EXIT_FAILURE );
    }
    sleep(1);

    if( clock_gettime( CLOCK_REALTIME, &stop) == -1 ) {
      perror( "clock gettime" );
      exit( EXIT_FAILURE );
    }

    accum = ( stop.tv_sec - start.tv_sec )
          + ( stop.tv_nsec - start.tv_nsec )
            / BILLION;
    printf( "%lf\n", accum );
    return( EXIT_SUCCESS );
}

So, lets start testing.

  1. clang -Wall -o gettime gettime.c gives no errors and producing working binary. We can test minimal supported version with otool -l gettime|grep -A 4 'LC_BUILD_VERSION' – in our case it is 10.14.
  2. Lets try to restrict version with 10.9. Command clang -mmacosx-version-min=10.9 -Wall -o gettime gettime.c silently producing output and gettime binary! Minimal osx version is now marked as 10.9. That is weird, but nm -a gettime shows reference to _clock_gettime symbol which should not exist on 10.9 at all! As expected, this binary fails if running on OSX 10.11:
        ./gettime
        dyld: lazy symbol binding failed: Symbol not found: _clock_gettime
          Referenced from: /Users/vagrant/./gettime
          Expected in: /usr/lib/libSystem.B.dylib
        dyld: Symbol not found: _clock_gettime
          Referenced from: /Users/vagrant/./gettime
          Expected in: /usr/lib/libSystem.B.dylib
        Trace/BPT trap: 5
    

    So despite the switch clang created broken binary!

  3. It is possible to disable weak imports by using -no_weak_imports linker flag. Lets try it:

    clang -Wl,-no_weak_imports -mmacosx-version-min=10.9 -Wall -o gettime gettime.c
    

    will fail to link with an error:

    ld: weak import of symbol '_clock_gettime' not supported because of option: -no_weak_imports for architecture x86_64.
    

Sounds like a solution! Moreover – we can add -Werror=partial-availability key and if it is used – compilation will fail early, with gettime.c:17:9: error: 'clock_gettime' is only available on macOS 10.12 or newer [-Werror,-Wunguarded-availability] error which is explaining where the problem is. So, looks like we can just add -Werror=partial-availability to C(XX)FLAGS and -Wl,-no_weak_imports to the link flags to solve all the issues?
Not so easy 😦

Testing with autoconf or cmake

In the real world software is typically built using some build system, which trying to detect what is available on the host to set defines accordingly. In the opensource world autoconf and cmake are very popular choices. Now lets try to see if we can build some projects with a compatibility flags set to 10.9. As a real-world example i will use XZ Utils which uses autoconf as a build system.

  1. Lets try to build XZ Utils 5.2.4 with default CFLAGS/LDFLAGS. Typical ./configure && make command should provide binary src/xz/.libs/xz with minimal supported OS set to 10.14. But out goal is to compile it for 10.9+, so lets do make clean
  2. Now lets try to set LD/CFLAGS to enforce 10.9 compatibility:
    export CFLAGS="-mmacosx-version-min=10.9 -Werror=partial-availability"
    export LDFLAGS="-Wl,-no_weak_imports"
    

    and re-run ./configure && make. This time build will fail with ../../src/common/mythread.h:250:19: error: '_CLOCK_REALTIME' is only available on macOS 10.12 or newer [-Werror,-Wunguarded-availability] error message.

Looking in the source it is clear that HAVE_CLOCK_GETTIME is set despite the fact that we used correct compiler and linker flags. Why it happens? Due to autoconf detection algorithm. To detect if function is available its trying to compile and link such simple code:

#ifdef __cplusplus
extern "C"
#endif
char clock_gettime ();
int
main ()
{
return clock_gettime ();
  ;
  return 0;
}

As you could see – autoconf includes own function prototype (because it only wants to check that function exists in the system) and in this case all linker/preprocessor version selection logic in clang/osx will just never run because it depends on definitions from the SDK headers.

Lets check this in the command line:

clang -mmacosx-version-min=10.9 -Werror=partial-availability -Wl,-no_weak_imports -o test test.c

will return no errors, but resulted binary will link to the _clock_gettime symbol which will be not available on target OS. I found that cmake checks are behaving exactly the same, resulting with a broken binary or compilation/link error if appropriate flags are set.

Workarounds

If it is your own code which does not have million dependencies – probably easiest way is to just patch autoconf/cmake checks to not prototype this functions but use system headers instead. However, if you have a tonn of dependencies – it would take huge amount of time to patch every failed one. So i found another workaround.

If recent (10.14) SDK is not easy to use in out scenario with autoconf/cmake lets use one from 10.9 🙂 I downloaded and unpacked 10.9 SDK to the /Library/Developer/CommandLineTools/SDKs/MacOSX10.9.sdk directory. I found that it is enough to set SDKROOT environment variable to the MacOSX10.9.sdk directory to fix the problem. In this case SDK will not contain any definitions of the non-compatible symbols and autoconf/cmake checks will work as expected, so you will get 10.9+ compatible Mach-O binary in the end of the game.

Summary

It is possible, but not trivial to keep compatibility with an older systems with a recent clang/OSX.

If you know better solution or workaround or just found this post useful – please let me know in the comments.

Advertisements
Tagged , ,

MacBook Pro USB-C 10G interconnection

I just found that it is possible to connect 2 2016 Macbook Pro laptops with an USB-C cable. Just in case – one from power adapter will not work, but another one from the Lenovo monitor works just fine. Connection is named as Thunderbolt Bridge in the network preferences.

There is no connection speed in the ifconfig output (just “active” mark), so decided to measure it with iperf3 – it shows about 4.20 Gbits/sec with default MTU on TCP test. And when i set MTU to 9000 i finally got 10.4 Gbits/sec, what is amazing 🙂 So now i know fastest and very easy method to transfer data between new macbooks just by using usb-c data cable. Rare moments when i really like Apple engineering 🙂

Tagged , ,

Smartmontools SVN builds

Smartmontools SVN builds (Linux, Windows, FreeBSD and OSX) are now available at https://builds.smartmontools.org/. Builds are done using docker and CircleCI and should be 100% repeatable. Keep in mind that these builds are not releases – some functionality could be broken or not yet completed.

Tagged , , ,

GIMP 2.10.6/OSX port released

You can download latest GIMP 2.10.6/OSX build from the github project page. Feel free to report any bugs. It will become an official download soon.

Tagged , ,

First release of the Gimp 2.10.2/OSX port

There is no [yet] official dmg package of the Gimp Open Source Image Editor for OSX, so i created a new one. Work is mostly done, there are few non-critical issues i am planning to fix meantime. Also my plans are to merge my patches with an upstream, so hopefully this would become official port at some point.

Testers are welcome. OSX build could be found on project releases page.

Tagged , ,

How my Mac Book Pro 2017 destroyed 2 USB-C devices

Bad things happens

At morning my new Macbook was not working well – when i been disconnecting charger it was not detecting this properly, so i decided to reset SMC, and this resolved the problem. In the office i connected my passive USB-C hub with HDMI/USB output and it starts to smell very bad. I was stupid enough to connect my USB-C ubikey to the same socket and it was destroyed immediately, with the same smell and some lights. Both devices were destroyed. MacBook was working like nothing happens.

I realized that i already saw post about it on Reddit and been able to find original post Macbook Pro frying USB peripherals. It was looking exactly as my case. So i decided to validate it.

Measuring USB-C power voltage

I do have usb-c->microusb adapter from some charger + microusb power to power adapter plug. I connected to cables to it and been able to measure the voltage, thats how it looks:

IMG_2982

Now i connected it to the voltmeter and plugged in to the affected port with a usb-c charger plugged in a different port:
IMG_8925

20V!!! instead of maximum 5 allowed. All other sockets were provided ~1v, probably that is idle current when no data lines are connected:
IMG_4172

I tried different combinations, but only top right socket was failing. So it seems to be another “known issue” in this new model, in addition to very problematic keyboard.

Summary

I been using only original charger with original cable but got usb-c devices fried. According to the post i provided below – i am not alone with such issue. Tomorrow i will send this Mac to repair and hopefully replacement will not be affected. I think that Apple engineers should consider this issue as very serious one, but giving the fact how they handle keyboard-related issues it seems to be unlikely.

Tagged , , ,

Fixing Google Music Manager on OSX

Sometime i am purchasing music on Google Music. Once i decided to download entire connection
and found that its not as easy as i exptected.

  1. With a browser you can download only album by album + it warns that download
    counter is limited to 2 times only (WTF?!?)
  2. Recommended way to download is “Google Music Manager” which should do everything automatically.

Okay google, lets try to install it. It installs fine, but download failing completely. On the web i found
that it was that way > 1 year already! (e.g. see this thread.

Also i tried to use Google support and it was as useless as possible.

So there is nothing to do but to try to fix it myself.

Starting debugging

Music Manager process name is MusicManagerHelper. This is QT4 (!) application.
Output of the lsof command shows that there is a log in the ~/Library/Logs/MusicManager/MusicManagerHelper.log.
Output shows that there is an error with certificate validation on every download attempt:

Failed to Verify Certificate: /C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.googleusercontent.com (unable to get local issuer certificate:0x00000000) [/Volumes/Android/buildbot/src/googleplex-android/jumper-stable/jumper/source/Shared/HttpClients/CurlHttpClientImpl.cpp:44 ::CurlSslVerifyCertificate()]

Ok, its better. Something i tried without any luck:

  • Adding google certs to the system keychain with a trust settings
  • Trying to find where its storing root certs (its compiled in)
  • Trying to rebuild libQtNetwork 4 on OSX (seems to be completely broken with a recent compiler/sdk)

Finally, solving issue using LLDB

Using debugger i found that libQtNetwork contains statically compiled OpenSSL.
After all i been able to identify function responsible for the certificate validation – it is ssl_verify_cert_chain.
Lets try to use lldb to override it return value!

  1. Starting lldb with lldb -p
  2. Adding breakpoint with breakpoint set -n ssl_verify_cert_chain --shlib libQtNetwork.4.dylib command.
  3. Attaching python script to the breakpoint:
    breakpoint command add –script-type python

    print "starting"
    thread = frame.GetThread()
    return_value =  lldb.frame.GetValueForVariablePath("*self")
    thread.ReturnFromFrame(frame, return_value)
    print "ending"
    lldb.debugger.GetSelectedTarget().process.Continue()
    DONE
    
  4. Trying to download again. LLDB should print “starting” and “ending” on
    any download and it finally works!

Afterwords

  • Google is an Evil. I spent a lot of time to just download content i paid for! And support was completely useless.
  • Google Music Manager, at least on OSX, is insecure, broken and outdated application.
  • Putting root certificates to the bundle is a very poor coding style.
    Not sure if it comes from QT or from Google itself, but doing that way is embedding timebomb to your app.
  • Dtrace and LLDB are cool 🙂
Tagged , ,

C++ cross-compilation on Linux for the FreeBSD target using GCC or CLANG

Overview

Sometime it could be convenient to generate FreeBSD binaries on the Linux host. Just a few examples:

  • CI/CD deployment with a compilation check on the Linux Containers. It is especially the case for the SaaS environments where you are not able to select another OS (or it is not supported by the vendor)
  • Ability to automatically generate binaries for the all supported platforms with a minimal cost/efforts.
  • Using single docker or other linux-specific technology to cover all platforms.

I am using osxcross without any problem to generate OSX binaries, so decided to do the same with FreeBSD. This article shows how to do this and will provide some automated examples.

Using GCC

Most of the articles in the net i found are recommending to use GCC, which is default compiler on Linux distributions. GCC supports cross-compilation natively, but to achieve this you need to build entire kit from scratch.

Short overview of the process:

  1. Compile install latest binutils with --target=x86_64-pc-freebsd10 configure argument.
  2. Get FreeBSD libs/includes and install them to the buildroot location
  3. Compile and install GMP with --host=x86_64-pc-freebsd10 configure argument.
  4. Compile and install MPC with --host=x86_64-pc-freebsd10 configure argument.
  5. Finally compile GCC, you will have to specify sysroot path and all the libs location. Also you will have to manually define #define HAVE_ALIGNED_ALLOC 1 due to bug in the GCC autoconf scripts.

I created Dockerfile with all steps mentioned. GCC natively supporting only libstdc++ which is not available on the FreeBSD. Below is an example:

root@96de1e64fff1:/# /opt/cross-freebsd/bin/x86_64-pc-freebsd10-c++ -o helo helo.cpp
root@96de1e64fff1:/# file helo
helo: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 10.3, not stripped
root@96de1e64fff1:/#  readelf -d helo|grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]

You can override this by overriding this using compiler/linker flags:

root@96de1e64fff1:/# /opt/cross-freebsd/bin/x86_64-pc-freebsd10-c++ -std=c++11 -I/opt/cross-freebsd/x86_64-pc-freebsd10/usr/include/c++/v1 -nostdinc++ -nodefaultlibs -lc++ -lm -lc -lgcc_s -lgcc -o helo helo.cpp

root@96de1e64fff1:/# readelf -d helo|grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libc++.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libcxxrt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]

Some comments on the commandline:

  • -std=c++11 -nostdinc++ needed to avoid errors on libc++ headers
  • -nodefaultlibs -lc++ -lm -lc -lgcc_s -lgcc will tell gcc to not use with a predefined libraries but to use libc++ instead.

Problems with GCC cross-build:

  • Entire build chain need to be built from scratch. Not a big issue if you are using containers, but still takes some time to maintain it.
  • Natively only libstdc++ is supported which is now not part of the FreeBSD installation. To workaround this you can provide it with your code or to build static. It is possible to use libc++ instead but this required a lot of custom flags and may not work reliable.
  • Natively FreeBSD is not using gcc (with only exception of some ports), so behavior of the resulted code may be different due to some compiler differences.

Using Clang

After all i decided to take another approach. Clang is now (starting from FreeBSD 10) default compiler, as well as libc++ is used instead of libstdc++. Great news is that clang already supports cross-compilation without need to completely recompile entire toolset. I been able to get working FreeBSD binaries on default Ubuntu LTS buildhost.

Below is a part of the Dockerfile to fetch required headers and libraries, install them to /opt/cross-freebsd-10 and fix broken links:

# Get FreeBSD libs/headers, extract and fix broken links
RUN cd /tmp &amp;&amp; wget http://ftp.plusline.de/FreeBSD/releases/amd64/10.4-RELEASE/base.txz \
&amp;&amp; mkdir -p /opt/cross-freebsd-10 \
&amp;&amp; cd /opt/cross-freebsd-10 \
&amp;&amp; tar -xf /tmp/base.txz ./lib/ ./usr/lib/ ./usr/include/ \
&amp;&amp; cd /opt/cross-freebsd-10/usr/lib \
&amp;&amp; find . -xtype l|xargs ls -l|grep ' /lib/' \
| awk '{print &quot;ln -sf /opt/cross-freebsd-10&quot;$11 &quot; &quot; $9}' \
| /bin/sh &amp;&amp; \
rm -f /tmp/base.txz

Now it is possible to create FreeBSD binaries using additional compiler flags: -target x86_64-unknown-freebsd10.0 --sysroot=/opt/cross-freebsd-10/

E.g.

root@4f9b820ae1c9:/# clang++ -target x86_64-unknown-freebsd10.0 --sysroot=/opt/cross-freebsd-10/ -o helo helo.cpp
root@4f9b820ae1c9:/# file helo
helo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 10.4, not stripped

Resulted binary will run on FreeBSD host only.

Lets check dependencies:

root@4f9b820ae1c9:/# readelf -d helo|grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libc++.so.1]
0x0000000000000001 (NEEDED) Shared library: [libcxxrt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libm.so.5]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.7]

As we could see dependencies are fine and matching normal FreeBSD 10.4 installation. Of course you may have many buildroots to simplify multiply targets.

Conclusion

It was found that both GCC and CLANG are able to cross-compile to FreeBSD using Linux host. However, CLANG require less efforts for this, mostly due to implemented -target switch and natively supported libc++ used by the FreeBSD project. I think it is better to use it for this task, with only exception if your source code has some strict GCC dependencies.

I implemented cross compilation to the FreeBSD as part of the CircleCI tests for the smartomontools project. I am planning to play with other targets (ARM, PPC) as well.

Tagged ,

Network FreeBSD boot using Mikrotik router TFTP

Sometime bad things happens

One of my FreeBSD servers failed to boot after number of power outages on the
remote site. However IPMI was still working, so I been able to see remote state.

Problem was with damaged ZFS root pool, so single user mode was also not able to boot.

At this location i also had Mikrotik router connected to the same switch, so i
decided to try to fix this server using network (PXE) boot.
It was a challenge for me, because i never tried to boot from the
RouterOS device and was not sure if it will work at all.

Configuring Mikrotik DHCP server

To get network boot possible there are 2 minimal pre-requirements:

  • DHCP server to provide DHCP responses with a special parameters
    (file to boot).
  • TFTP server to host files required for network boot.

Normally FreeBSD also wants NFS for the remote booting, but in this case we will avoid it.

To configure DHCP on the Mikrotik i created new DHCP server and dedicated DHCP network for it, which includes boot-file-name property:

/ip dhcp-server
add interface=MYLAN name=tftpsever

/ip dhcp-server network
add address=10.0.31.0/24 boot-file-name=gpxelinux.0 dns-server=10.0.31.1 \
    gateway=10.0.31.1 netmask=24

Also static lease for the server ethernet card MAC address been added:

/ip dhcp-server lease
add address=10.0.31.30 always-broadcast=yes mac-address=\
      E4:1F:13:6C:12:34 server=tftpsever

Also i would recommend to enable DHCP logging to monitor status of the requests.

Configuring Mikrotik TFTP server

When DHCP configuration is completed, it is time to enable and configure TFTP on
the router. In the RouterOS you have to specify parameters for every hosted file.

Below is my configuration:

/ip tftp
add real-filename=gpxelinux.0 req-filename=gpxelinux.0
add real-filename=chain.c32 req-filename=chain.c32
add real-filename=pxelinux.cfg/default req-filename=\
    pxelinux.cfg/default
add real-filename=memdisk req-filename=memdisk
add real-filename=menu.c32 req-filename=menu.c32
add real-filename=reboot.c32 req-filename=reboot.c32
add allow-rollover=yes real-filename=\
    mfsbsd-11.0-RELEASE-amd64.img req-filename=mfsbsd-11.0-RELEASE-amd64.img

Creating bootable mfsbsd installation on the TFTP server

To recover FreeBSD i decided to use mfsbsd project. This is FreeBSD live CD/USB
which loads to the memory (so no NFS needed) and provides a lot of useful tools
out of the box. Also due to it small size it was possible to fit it on the router flash memory, so no additional downloads or mounts were needed. And, of course, mfsbsd contains everything needed to repair broken ZFS pool. I been using USB image based on 11.0 RELEASE.

But mfsbsd alone is not enough – to boot from network we also need files from the syslinux project. I been using [outdated] version 4.04, just because i had some projects using it in the past and it seems that some modules were renamed in the recent version.

List of the files to upload:

gpxelinux.0 (syslinux-4.04/gpxe/gpxelinux.0)
chain.c32 (syslinux-4.04/com32/modules/chain.c32)
memdisk (syslinux-4.04/memdisk/memdisk)
menu.c32 (syslinux-4.04/com32/menu/menu.c32)
reboot.c32 (syslinux-4.04/com32/modules/reboot.c32)
mfsbsd-11.0-RELEASE-amd64.img (USB image from the mfsbsd web site)
pxelinux.cfg/default (configuration, see below)

To work properly with IPMI and boot mfsbsd image we need to create configuration. Line serial 1 115200 is to define serial port output, and menu ui is mostly to ensure that something useful is going on.

Content of the pxelinux.cfg/default file:

serial 1 115200
console 1
ui menu.c32
menu title Utilities

label mfsbsd
  menu label mfsBSD
  kernel memdisk
  initrd mfsbsd-11.0-RELEASE-amd64.img harddisk raw

Files needs to be copied to the router flash, i used sftp tool to upload them.

Configuring server

Make sure that you have enabled PXE boot in the BIOS (i been able to do this
using IPMI). Also, at least on my server, i had to enable not only Legacy PXE
(which is in use), but also UEFI IPMI or network boot was never starting.
If everything is done correctly you should see DHCP request followed by TFTP.
In my case it was 2 times:

  1. UEFI PXE, which failed because i am using legacy boot.
  2. Legacy PXE which succeed. After loading all required files boot menu was
    shown and i been able to select mfsbsd item.
  3. After selecting this item FreeBSD started to load, but no output been shown
    on IPMI console. This is because by default mfsbsd is not showing anything on the
    serial console. It should be easy to fix, but in my case it was not needed.

In a few minutes i found another DHCP request in the Mikrotik logs, this time
from the mfsbsd, and been able to reach the server using SSH (credentials are root/mfsbsd). It is possible to use RouterOS ssh client (/system ssh user=root 10.0.31.30).

Finally fixing ZFS pool

My attempt to import failed zroot pool from the mfsbsd failed as well. Moreover –
zpool import -F zroot also been not working. Before giving up i been able to find non-documented -X switch, which together with -F been able to completely restore pool in ~30 minutes! After reboot server been able to boot normally.

Todo

  • Check what needs to be done to create UEFI compatible FreeBSD network boot.
  • Create additional menu items for RescueCD.
  • Create mfsbsd which sends output to the serial port 2, to be compatible with IPMI.
  • Utilize latest syslinux instead of very outdated one.
Tagged , , ,

How to monitor NVMe drives in the OSX

NVMe support in OSX

After upgrade to the latest Macbook Pro i found that smartctl is not able to find any smart capable drive. This is because Apple replaces SATA SSD with NVMe one and old SMART API is not working (it is very ATA specific). Smartmontools itself includes NVMe support for the Linux, Windows and FreeBSD, so i decided to try to add it to the Darwin as well. However it was not as easy as expected – Apple did not published any source code or documentation about NVMe device support or monitoring. Moreover – there is no any tool in OSX to show such statistic and old tools from SDK are useless because of API Change.

Starting to search for the API provider

After looking on the file tree i have found good candidate: /System/Library/Extensions/NVMeSMARTLib.plugin. Its done more or less similar to the /System/Library/Extensions/SMARTLib.plugin/ which provides SATA/ATA SMART support. As i mentioned – there is no documentation, so i had to use otool, nm and lldb to deal with it. As expected, it was found that API is similar to the ATA one. You can get list of the symbols and functions using this command: nm NVMeSMARTLib | c++filt -p -i. So i tried to connect to it using modified example from SDK for the SMART. Tricky part was to find kIONVMeSMARTUserClientTypeID and kIONVMeSMARTInterfaceID values which are using by CFPLUGIN infrastructure in the IOKit to initialize API interface. Fortunately library comparing this data during runtime, so with disasm i been able to find them. After successful connect to the interface i been able to reconstruct missing headers and use some of the functions (see below)

What is working and what is not.

The most important functions SMARTReadData and GetIdentifyData are working and result is provided in the structures matching with NVMe standard. I was not able to get GetLogPage function running, probably it expects pointer to some structure with defined data. If apple will release any consumer of it it would be easy to find this out.

Also there are some other, unknown functions in this API: GetFieldCounters (always returns error), ScheduleBGRefresh (no parameters, returns ok), GetSystemCounters and GetAlgorithmCounters (some driver info? or vendor-specific log pages?). Another interesting finding was string “Sandisk 401Z128G-4p-MLC” in the SMART log page, so possibly this NVMe is originally from this vendor.

SmartMontools support

I am working to add limited NVMe support to the smartctl and smartd for OSX. Now i already have working prototype, but need to cleanup and refactor some code. I am planning to add this before the next release. Below is an output from my disk:

smartctl 6.6 2017-09-14 r4434M [Darwin 16.7.0 x86_64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===

Model Number:                       APPLE SSD AP0512J
Serial Number:                      XXXXXX
Firmware Version:                   16.14.01
PCI Vendor/Subsystem ID:            0x106b
IEEE OUI Identifier:                0x000502
Controller ID:                      0
Number of Namespaces:               2
Local Time is:                      Wed Sep 20 08:56:36 2017 CEST
Firmware Updates (0x02):            1 Slot
Optional Admin Commands (0x0004):   Frmw_DL
Optional NVM Commands (0x0004):     DS_Mngmt
Maximum Data Transfer Size:         256 Pages

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     0.00W       -        -    0  0  0  0        0       0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0x0)
Critical Warning:                   0x00
Temperature:                        33 Celsius
Available Spare:                    90%
Available Spare Threshold:          2%
Percentage Used:                    0%
Data Units Read:                    19,311,330 [9.88 TB]
Data Units Written:                 11,653,167 [5.96 TB]
Host Read Commands:                 50,388,833
Host Write Commands:                37,404,327
Controller Busy Time:               0
Power Cycles:                       2,320
Power On Hours:                     23
Unsafe Shutdowns:                   7
Media and Data Integrity Errors:    0
Error Information Log Entries:      0

Reconstructed API

If you want to play with the API yourself – you can use this header. Please let me know if you found how to use GetLogPage or any other useful information:

// NVMe definitions, non documented, experimental

// Constant to init driver
#define kIONVMeSMARTUserClientTypeID       CFUUIDGetConstantUUIDWithBytes(NULL,      \
                                        0xAA, 0x0F, 0xA6, 0xF9, 0xC2, 0xD6, 0x45, 0x7F, 0xB1, 0x0B, \
                    0x59, 0xA1, 0x32, 0x53, 0x29, 0x2F)

// Constant to use plugin interface
#define kIONVMeSMARTInterfaceID        CFUUIDGetConstantUUIDWithBytes(NULL,                  \
                    0xcc, 0xd1, 0xdb, 0x19, 0xfd, 0x9a, 0x4d, 0xaf, 0xbf, 0x95, \
                    0x12, 0x45, 0x4b, 0x23, 0xa, 0xb6)

// interface structure, obtained using lldb, could be incomplete or wrong
typedef struct IONVMeSMARTInterface
{
        IUNKNOWN_C_GUTS;

        UInt16 version;
        UInt16 revision;

                // NVMe smart data, returns nvme_smart_log structure
        IOReturn ( *SMARTReadData )( void *  interface,
                                     struct nvme_smart_log * NVMeSMARTData );

                // NVMe IdentifyData, returns nvme_id_ctrl per namespace
        IOReturn ( *GetIdentifyData )( void *  interface,
                                      struct nvme_id_ctrl * NVMeIdentifyControllerStruct,
                                      unsigned int ns );

                // Always getting kIOReturnDeviceError
        IOReturn ( *GetFieldCounters )( void *   interface,
                                        char * FieldCounters );
                // Returns 0
        IOReturn ( *ScheduleBGRefresh )( void *   interface);

                // Always returns kIOReturnDeviceError, probably expects pointer to some
                // structure as an argument
        IOReturn ( *GetLogPage )( void *  interface, void * data, unsigned int, unsigned int);


                /* GetSystemCounters Looks like a table with an attributes. Sample result:

                0x101022200: 0x01 0x00 0x08 0x00 0x00 0x00 0x00 0x00
                0x101022208: 0x00 0x00 0x00 0x00 0x02 0x00 0x08 0x00
                0x101022210: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x101022218: 0x03 0x00 0x08 0x00 0xf1 0x74 0x26 0x01
                0x101022220: 0x00 0x00 0x00 0x00 0x04 0x00 0x08 0x00
                0x101022228: 0x0a 0x91 0xb1 0x00 0x00 0x00 0x00 0x00
                0x101022230: 0x05 0x00 0x08 0x00 0x24 0x9f 0xfe 0x02
                0x101022238: 0x00 0x00 0x00 0x00 0x06 0x00 0x08 0x00
                0x101022240: 0x9b 0x42 0x38 0x02 0x00 0x00 0x00 0x00
                0x101022248: 0x07 0x00 0x08 0x00 0xdd 0x08 0x00 0x00
                0x101022250: 0x00 0x00 0x00 0x00 0x08 0x00 0x08 0x00
                0x101022258: 0x07 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x101022260: 0x09 0x00 0x08 0x00 0x00 0x00 0x00 0x00
                0x101022268: 0x00 0x00 0x00 0x00 0x0a 0x00 0x04 0x00
                .........
                0x101022488: 0x74 0x00 0x08 0x00 0x00 0x00 0x00 0x00
                0x101022490: 0x00 0x00 0x00 0x00 0x75 0x00 0x40 0x02
                0x101022498: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                */
        IOReturn ( *GetSystemCounters )( void *  interface, char *, unsigned int *);


                /* GetAlgorithmCounters returns mostly 0
                0x102004000: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004008: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004010: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004018: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004020: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004028: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004038: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004040: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004048: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004050: 0x00 0x00 0x00 0x00 0x80 0x00 0x00 0x00
                0x102004058: 0x80 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004060: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004068: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004070: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004078: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004080: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004088: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004090: 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00
                0x102004098: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

                */
        IOReturn ( *GetAlgorithmCounters )( void *  interface, char *, unsigned int *);
} IONVMeSMARTInterface;


Tagged , , , ,