I told the Microsoft Visual C++ compiler not to generate AVX instructions, but it did it anyway!

https://devblogs.microsoft.com/oldnewthing/20201026-00/?p=104397

https://devblogs.microsoft.com/oldnewthing/?p=104397

A customer passed the /arch:SSE2 flag to the Microsoft Visual C++ compiler, which means “Enable use of instructions available with SSE2-enabled CPUs.” In particular, the customer did not pass the /arch:SSE4 flag,¹ so they did not enable the use of SSE4 instructions.

And then they did this:

#include <mmintrin.h>

void something()
{
    __m128i v = _mm_load_si128(&mem);
    ... more SSE2 stuff ...
    v = _mm_insert_epi32(v, alpha, 3);
    ... more SSE2 stuff ...
}

The _mm_insert_epi32() intrinsic maps to the PINSRD instruction, which is an SSE4 instruction, not SSE2.

To the customer’s surprise, this code not only compiled, it even ran! The customer wanted to know what is happening. Did the compiler convert the _mm_insert_epi32() into an equivalent series of SSE2 instructions?

No, the compiler didn’t do that. You explicitly requested an SSE4 instruction, so the compiler honored your request. The /arch:SSE2 flag tells the compiler not to use any instructions beyond SSE2 in its own code generation, say during autovectorization or optimized memcpy. But if you invoke it explicitly, then you get what you wrote.

I guess the option could be more accurately (and verbosely) named “Enable automatic use of instructions available with SSE2-enabled CPUs.” Because what this controls is whether the compiler will use those instructions of its own volition.

The customer happened to test their program on a CPU that supported SSE4, so the instruction worked. If they had run it on a a CPU that supported SSE2 but not SSE4, it would have crashed.

The reason SSE4 intrinsics are still allowed even in SSE2 mode is that you might have identified some performance-sensitive operations and written two versions of the code, one that uses SSE2 intrinsics, and another that uses SSE4 intrinsics, choosing between the two at runtime based on a processor capability check.

The compiler won’t generate any SSE4 instructions on its own, so your code is safe on SSE2 systems. When you detect an SSE4 system, you can explicitly call the SSE4 code paths.

¹ As commenter Danielix Klimax noted, there is no actual /arch:SSE4 option. Please interpret the remark in the spirit it was intended. (“The custom did not pass any flags that would enable SSE4 instructions.”)

The post I told the Microsoft Visual C++ compiler not to generate AVX instructions, but it did it anyway! appeared first on The Old New Thing.

The BurgerMaster segment may have been legendary, but some legends aren’t true

https://devblogs.microsoft.com/oldnewthing/20201027-00/?p=104399

https://devblogs.microsoft.com/oldnewthing/?p=104399

Some time ago, I discussed the historical significance of the Burgermaster drive-in restaurant. This triggered a memory from one of my colleagues:

I also heard that the BurgerMaster was identified by an ordinal value, and that ordinal value was the telephone number of the restaurant.

That sounds like a fun story. Alas, it doesn’t hold up.

The Get­Proc­Address function checks the numeric value of the string pointer it is given. If it is greater than or equal to 65536, then it is interpreted as a string pointer. But if it is less than 65535, then it’s treated as an ordinal. Therefore, ordinal values are limited to 65535. That’s not enough digits to hold a U.S. telephone number.

The Global­Master­Handle function was exported as ordinal 28. There does not appear to be any significance to this number. It was just the next available ordinal in the block assigned to global memory functions.

The post The BurgerMaster segment may have been legendary, but some legends aren’t true appeared first on The Old New Thing.

How can I tell whether a file is on an SSD?

https://devblogs.microsoft.com/oldnewthing/20201023-00/?p=104395

https://devblogs.microsoft.com/oldnewthing/?p=104395

You might want your program to change its behavior depending on whether the file you are operating on is on an SSD or not. Maybe you’d use Prefetch­Virtual­Memory to get the contents of a memory-mapped file into memory more efficiently if the file is on a hard drive, but not bother if the file is on an SSD, since the SSD can produce the data quickly enough anyway.

bool IsFileOnSsd(PCWSTR filePath)
{
  wil::unique_hfile volume = GetVolumeHandleForFile(filePath);

  STORAGE_PROPERTY_QUERY query{};
  query.PropertyId = StorageDeviceSeekPenaltyProperty;
  query.QueryType = PropertyStandardQuery;
  DWORD bytesWritten;
  DEVICE_SEEK_PENALTY_DESCRIPTOR result{};

  if (DeviceIoControl(volume.get(), IOCTL_STORAGE_QUERY_PROPERTY,
      &query, sizeof(query),
      &result, sizeof(result),
      &bytesWritten, nullptr)) {
    return !result.IncursSeekPenalty;
  }
  return false;
}

This takes advantage of the trick we learned last time where you can make a storage query against a volume, and it will report the answer if the volume has a single extent.

We aren’t so much checking whether it’s on an SSD drive as we are checking whether seeks are free. That is true for SSDs, but it’s also true for RAM drives. But RAM drives are even faster than SSDs, so I think it’s okay to treat them as “super-awesome SSDs”.

The Get­Volume­Handle­For­File function we wrote a few days ago will throw if the file is remote (on a network). We probably want to report network files as “not on an SSD”, because even if they are on an SSD on the server, the network transmission cost will make it feel slow.

wil::unique_hfile GetVolumeHandleForFile(PCWSTR filePath)
{
  wchar_t volumePath[MAX_PATH];
  THROW_IF_WIN32_BOOL_FALSE(GetVolumePathName(filePath,
                                volumePath, ARRAYSIZE(volumePath)));

  wchar_t volumeName[MAX_PATH];
  if (!GetVolumeNameForVolumeMountPoint(volumePath,
                                volumeName, ARRAYSIZE(volumeName))) {
    return {};
  }

  auto length = wcslen(volumeName);
  if (length && volumeName[length - 1] == L'\\')
  {
    volumeName[length - 1] = L'\0';
  }

  wil::unique_hfile result{ CreateFile(volumeName, 0,
                FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                nullptr, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, nullptr) };
  THROW_LAST_ERROR_IF(!result);
  return result;
}

We would then check whether a volume was gotten:

bool IsFileOnSsd(PCWSTR filePath)
{
  wil::unique_hfile volume = GetVolumeHandleForFile(filePath);
  if (!volume) return false;

  STORAGE_PROPERTY_QUERY query{};
  query.PropertyId = StorageDeviceSeekPenaltyProperty;
  query.QueryType = PropertyStandardQuery;
  DWORD bytesWritten;
  DEVICE_SEEK_PENALTY_DESCRIPTOR result{};

  if (DeviceIoControl(volume.get(), IOCTL_STORAGE_QUERY_PROPERTY,
      &query, sizeof(query),
      &result, sizeof(result),
      &bytesWritten, nullptr)) {
    return !result.IncursSeekPenalty;
  }
  return false;
}

As we noted last time, the query against a volume will fail if the volume spans multiple physical disks. If you have a volume that spans multiple SSDs, this function will nevertheless report that it isn’t an SSD.

So we probably would be better off checking the SSD-ness of every physical drive in the volume. That’s a lot of work, so I’m going to cheat and check just the first physical drive in the volume, on the theory that when people create multi-drive volumes, they’re going to be drives of similar performance characteristics.

bool IsFileOnSsd(PCWSTR filePath)
{
  wil::unique_hfile volume = GetVolumeHandleForFile(filePath);
  if (!volume) return false;

  wil::unique_hfile disk =
    GetFirstPhysicalDiskHandleForVolume(volume.get());
  if (!disk) return false;

  STORAGE_PROPERTY_QUERY query{};
  query.PropertyId = StorageDeviceSeekPenaltyProperty;
  query.QueryType = PropertyStandardQuery;
  DWORD bytesWritten;
  DEVICE_SEEK_PENALTY_DESCRIPTOR result{};

  if (DeviceIoControl(disk.get(), IOCTL_STORAGE_QUERY_PROPERTY,
      &query, sizeof(query),
      &result, sizeof(result),
      &bytesWritten, nullptr)) {
    return !result.IncursSeekPenalty;
  }
  return false;
}

Bonus chatter: Many people cheat even further and also assume that the volume is mounted as a drive letter. In that case, obtaining the volume handle for the file is a simple matter of opening \\.\X:, where X: is the drive letter of the file you are interested in.

The post How can I tell whether a file is on an SSD? appeared first on The Old New Thing.

How do I get from a volume to the physical disk that holds it?

https://devblogs.microsoft.com/oldnewthing/20201021-00/?p=104387

https://devblogs.microsoft.com/oldnewthing/?p=104387

Last time, we saw how to get from a file path to the volume that holds it. The next step is to get from the volume to the physical disk.

The lazy way is to ask for the device number:

STORAGE_DEVICE_NUMBER number;
DeviceIoControl(handle,
    IOCTL_STORAGE_GET_DEVICE_NUMBER,
    nullptr, 0, // no input
    &number, sizeof(number), // output goes here
    &bytesWritten,
    nullptr);
DWORD physicalDriveNumber = number.DeviceNumber;

This is lazy for multiple reasons:

  • It fails to account for the case where the volume spans multiple physical drives.
  • In my experience, if the volume is a CD-ROM drive with no disk in the drive, the call reports that the physical drive number is 0, which is almost certainly incorrect.

In practice, it seems that if the volume spans multiple physical drives, the IOCTL_STORAGE_GET_DEVICE_NUMBER fails (with ERROR_INVALID_FUNCTION, it seems, which is the Win32 manifestation of the NT status code STATUS_INVALID_DEVICE_REQUEST), so at least you don’t get wrong answers. You just get no answer.

The less lazy (and more likely to be correct) way is to ask the volume for its disk extents. This one is a bit annoying because it returns a variable-sized structure, so you need to ask twice. The first time tells you how big a structure you need, and the second time actually gets the structure.

Since nearly all volumes have only one extent, we can optimize slightly for that case by passing an initial buffer big enough to hold a single extent. If that works, then there’s no need to try a second time.

VOLUME_DISK_EXTENTS* extents = nullptr;

// Anticipate common case where there is only one extent.
VOLUME_DISK_EXTENTS singleExtent;

// But also have a place to manage allocated data.
std::unique_ptr<BYTE[]> lifetime;

DWORD bytesWritten;
if (DeviceIoControl(handle, IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS,
        nullptr, 0,
        &singleExtent, sizeof(singleExtent),
        &bytesWritten,
        nullptr)) {
  // Worked on the first try. Use the preallocated buffer.
  extents = &singleExtent;
} else {
  VOLUME_DISK_EXTENTS* lastQuery = &singleExtent;
  while (GetLastError() == ERROR_MORE_DATA) {
    assert(RTL_CONTAINS_FIELD(lastQuery, bytesWritten, NumberOfDiskExtents));
    DWORD extentCount = lastQuery->NumberOfDiskExtents;
    DWORD allocatedSize = FIELD_OFFSET(VOLUME_DISK_EXTENTS, Extents[extentCount]);
    lifetime.reset(new BYTE[allocatedSize]);
    lastQuery = (VOLUME_DISK_EXTENTS*)lifetime.get();
    if (DeviceIoControl(handle, IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS,
          nullptr, 0,
          lastQuery, allocatedSize,
          &bytesWritten,
          nullptr)) {
      extents = lastQuery;
      break;
    }
  }
}

if (extents) {
  // process the extents
}

The extents tell you which physical drives the volume draws its storage from, and which bytes on those physical drives are devoted to the volume. But for this exercise, we just want the physical drives.

Once you have the physical drive numbers, you can convert them to physical drive handles by building a path of the form \\.\PhysicalDrive# where the # is the decimal expansion of the drive number.

wchar_t physicalDrivePath[80];
wsprintf_s(physicalDrivePath, L"\\\\.\\PhysicalDrive%d", physicalDriveNumber);
driveHandle = CreateFile(physicalVolumePath,
        0, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
        nullptr, OPEN_EXISTING, 0, nullptr);

Okay, great, now you have a physical drive handle.

Next time, we’ll see that there’s a shortcut available for all this.

Bonus chatter: If you are interested only in the first physical drive of a multi-drive volume, you can do it much more simply, because the ioctl will fill in as much of the buffer as it can. Passing a buffer that can hold one physical drive will give you the first physical drive. (Mind you, the drives don’t appear to be in any particular, order, so really, you’re just grabbing one at random.)

wil::unique_hfile GetFirstPhysicalDiskHandleForVolume(HANDLE volume)
{
  VOLUME_DISK_EXTENTS extents;
  if (!DeviceIoControl(volume, IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS,
        nullptr, 0,
        &extents, sizeof(extents),
        &bytesWritten,
        nullptr) && GetLastError() != ERROR_MORE_DATA) {
    THROW_LAST_ERROR();
  }

  wchar_t physicalDrivePath[80];
  swprintf_s(physicalDrivePath, L"\\\\.\\PhysicalDrive%u",
             extents.Extents[0].DiskNumber);

  wil::unique_hfile result{ CreateFile(physicalDrivePath, 0,
                      FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                      nullptr, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, nullptr) };
  THROW_LAST_ERROR_IF(!result);
  return result;
}

Bonus bonus chatter: It seems that the I/O subsystem can’t decide whether the number is a physical device number, a physical disk number, or a a physical drive number.

The post How do I get from a volume to the physical disk that holds it? appeared first on The Old New Thing.

Taking a shortcut: You can query properties from a volume, and it will forward to the physical drive

https://devblogs.microsoft.com/oldnewthing/20201022-00/?p=104391

https://devblogs.microsoft.com/oldnewthing/?p=104391

If you have the handle to a volume, you can issue certain disk ioctls to the volume, and it will forward them to the underlying disk. We saw this earlier when we used IOCTL_STORAGE_GET_DEVICE_NUMBER to obtain the physical drive number from a volume. The name of the ioctl is IOCTL_STORAGE, but we issued it against a volume anyway.

And as we saw earlier, if the volume does not have a unique physical disk, then the call will fail.

This feature is particularly handy with storage property queries. For example, you can ask what how the drive is connected to the system by querying the volume:

wil::unique_hfile volume = GetVolumeHandleForFile(L"C:\\");

STORAGE_PROPERTY_QUERY query{};
query.PropertyId = StorageAdapterProperty;
query.QueryType = PropertyStandardQuery;
DWORD bytesWritten;
STORAGE_ADAPTER_DESCRIPTOR result{};

if (DeviceIoControl(volume.get(), IOCTL_STORAGE_QUERY_PROPERTY,
    &query, sizeof(query),
    &result, sizeof(result),
    &bytesWritten, nullptr)) {
    /* result.BusType tells you how the drive is connected */
}

Next time, we’ll use this to answer a commonly-asked question.

The post Taking a shortcut: You can query properties from a volume, and it will forward to the physical drive appeared first on The Old New Thing.

How do I get from a file path to the volume that holds it?

https://devblogs.microsoft.com/oldnewthing/20201020-00/?p=104385

https://devblogs.microsoft.com/oldnewthing/?p=104385

Say you have the path to a file and you want to access the volume that the file resides on.

Warning: All error checking is removed for expository purposes.

The first step on our journey is getting from the path to the volume mount point. This tells us where the root of the volume got inserted into the namespace.

TCHAR volumePath[MAX_PATH]; // for expository purposes
GetVolumePathName(filePath, volumePath, ARRAYSIZE(volumePath));

This information might be useful in its own right, but for us, it’s just a stepping stone to the next piece of information: The volume name.

TCHAR volumeName[MAX_PATH]; // for expository purposes
GetVolumeNameForVolumeMountPoint(volumePath, volumeName, ARRAYSIZE(volumeName));

The volume name is returned in the form \\?\Volume{guid}\, with a trailing backslash. Note that this call will fail if the path is not a local drive.

Now things get weird.

If you pass that path to the CreateFile function with the trailing backslash intact, then you are opening a handle to the root directory of the volume.

If you pass that path to the CreateFile function with the trailing backslash removed, then you are opening a handle to the volume itself.

Depending on which operation you want to perform on the volume, you either must have or must not have that trailing backslash.

In our case, we want to open a handle to the volume itself, so we need to remove that trailing backslash. The call to CreateFile looks like this:

HANDLE handle = CreateFile(volumeNameWithoutTrailingBackslash,
    0, /* no special access requested */
    FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
    nullptr, /* no custom security */
    OPEN_EXISTING,
    FILE_FLAG_BACKUP_SEMANTICS,
    nullptr); /* template */

Volume query operations do not require any specific level of access, so we’ll ask for no special access. Which is good, because regular non-elevated code doesn’t really have much in the way of special access to volumes. You won’t able to get GENERIC_READ, much less GENERIC_WRITE. (You’ll be able to get FILE_READ_ATTRIBUTES, if that’s any consolation.)

You also need to request backup semantics in order to open a volume.

We can put all of this together into a function called, say, Get­Volume­Handle­For­File. For RAII, I’m going to use wil.

wil::unique_hfile GetVolumeHandleForFile(PCWSTR filePath)
{
  wchar_t volumePath[MAX_PATH];
  THROW_IF_WIN32_BOOL_FALSE(GetVolumePathName(filePath,
                                volumePath, ARRAYSIZE(volumePath)));

  wchar_t volumeName[MAX_PATH];
  THROW_IF_WIN32_BOOL_FALSE(GetVolumeNameForVolumeMountPoint(volumePath,
                                volumeName, ARRAYSIZE(volumeName)));

  auto length = wcslen(volumeName);
  if (length && volumeName[length - 1] == L'\\')
  {
    volumeName[length - 1] = L'\0';
  }

  wil::unique_hfile result{ CreateFile(volumeName, 0,
                FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                nullptr, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, nullptr) };
  THROW_LAST_ERROR_IF(!result);
  return result;
}

Now that you have the volume handle, you can ask the volume for information. We’ll look at that next time.

The post How do I get from a file path to the volume that holds it? appeared first on The Old New Thing.

Mount points, volumes, and physical drives, oh my!

https://devblogs.microsoft.com/oldnewthing/20201019-00/?p=104380

https://devblogs.microsoft.com/oldnewthing/?p=104380

At the bottom of the storage hierarchy are physical drives. These are units of physical storage, access to which is governed by a single disk controller. Your SSD or hard drive is a physical drive.¹

The next layer up is the volume. A volume is region of storage that is managed by a single file system. The relationship between volumes and physical drives is typically one-to-one, but it doesn’t have to be.

For example, a physical drive might not have any volume associated with it at all. For example, it could be a raw hard drive that hasn’t been partitioned or formatted yet.

You might take your physical drive and create multiple partitions, and then format each partition separately. Each of those formatted partitions is its own volume.

Or you might get really fancy and use a feature like spanned volumes or Storage Spaces to take multiple physical drives and combine them into one giant volume.

Once you have your volumes, you need to make them accessible somehow.²

Mount points are places that volumes are inserted into the namespace and become paths. The most usual place to see them is as a drive letter. For example, your system boot volume is almost certainly mounted as C:.

Volumes don’t have to be mounted as drive letters, though. You can also mount them inside a subdirectory of an existing volume, sort of like grafting one tree onto another. One way to do this is by going to the Disk Management tool, right-clicking a volume, and selecting Change drive letter and paths. From there, you can add a path for a volume, and the contents of the volume will be visible via that path.

Note that a volume can be mounted in multiple places, or it might not be mounted at all.

Next time, we’ll look at how to navigate these concepts in code.

¹ Paradoxically, you can have virtual physical drives, like a RAM drive.

² Theoretically, you could eschew mounting the volume and just access it via its volume GUID.

C:\>type \\?\Volume{8c3513a4-d064-4c99-81fc-66e20810ec3c}\windows\win.ini
; for 16-bit app support
[fonts]
[extensions]
[mci extensions]
[files]
[Mail]
MAPI=1

I mean, theoretically you could do that, but you’d also be a little bit crazy.

The post Mount points, volumes, and physical drives, oh my! appeared first on The Old New Thing.

Structured binding in C++/WinRT: The key-value pair

https://devblogs.microsoft.com/oldnewthing/20201016-00/?p=104371

https://devblogs.microsoft.com/oldnewthing/?p=104371

Last time, we learned how to add structured binding support to your own types, and noted that the get functions do not have to return references.

C++/WinRT has something similar to a std::pair: The IKeyValuePair<K, V>, which is used to represent a single entry in a map. (C++ uses a std::pair for this.)

Since the kvp.Key() and kvp.Value() methods always return by value, it means that when you use structured binding on an IKeyValuePair, the variables are always copies. The qualifiers on the auto merely describes how the kvp itself is captured.

IKeyValuePair<hstring, int32_t> kvp;
auto& [key, value] = kvp;

This looks like it’s binding key and value as lvalue references, but it’s not. They are non-reference variables. That’s because the code expands to

auto& hidden = kvp;
decltype(auto) key = kvp.Key();
decltype(auto) value = kvp.Value();

Since the results stored into key and value don’t depend on how you bound the source, you may as well bind the hidden variable by reference to avoid an unnecessary copy.

// wasteful copy from kvp to hidden
auto [key, value] = kvp;

// non-copying binding
auto&& [key, value] = kvp;

Bonus chatter: The structured binding of IKeyValuePair comes in particularly handy when you are iterating over something like an IMap:

for (auto&& [key, value] : map)
{
  // use key and value
}

The post Structured binding in C++/WinRT: The key-value pair appeared first on The Old New Thing.

How to add C++ structured binding support to your own types

https://devblogs.microsoft.com/oldnewthing/20201015-00/?p=104369

https://devblogs.microsoft.com/oldnewthing/?p=104369

Last time, we took a quick look at C++ structured binding. This time, we’ll see how to add structured binding support to your own types.

For concreteness, let’s say that we want to make this class available to structured binding:

class Person
{
public:
  std::string name;
  int age;
};

Now, technically, you don’t have to do anything to make this available to structured binding because there are special rules that automatically enable structured binding for simple structures. But let’s do it manually, just so we can see how it’s done.

Step 1: Include <utility>.

Step 2: Specialize the std::tuple_size so that its value is a std::size_t integral constant that says how many pieces there are.

In our case, we have two pieces, so the value is 2.

namespace std
{
  template<>
  struct tuple_size<::Person>
  {
    static constexpr size_t value = 2;
  };
}

If you have included <type_traits>, then you can use the predefined integral_constant template class to do the work of declaring the value.

namespace std
{
  template<>
  struct tuple_size<::Person>
      : integral_constant<size_t, 2> {};
}

Step 3: Specialize the std::tuple_element so that it identifies the type of each piece. You need as many specializations as you have pieces you declared in Step 2. The indices start at zero.

namespace std
{
  template<>
  struct tuple_element<0, ::Person>
  {
    using type = std::string;
  };

  template<>
  struct tuple_element<1, ::Person>
  {
    using type = int;
  };
}

If you have only two parts, you can simplify this by taking advantage of std::conditional.

namespace std
{
  template<size_t Index>
  struct tuple_element<Index, ::Person>
    : conditional<Index == 0, std::string, int>
  {
    static_assert(Index < 2,
      "Index out of bounds for Person");
  };
}

If you have more than two pieces, I guess you could chain the conditionals:

namespace std
{
  template<size_t Index>
  struct tuple_element<Index, ::Whatever>
    : conditional<Index == 0, std::string,
        conditional<Index == 1, int, whatever>>
  {
    static_assert(Index < 3,
      "Index out of bounds for Whatever");
  };
}

but this gets unwieldy really fast. I would repurpose std::tuple, which we saw some time ago was a handy way to store a bunch of types.

namespace std
{
  template<size_t Index>
  struct tuple_element<Index, ::Whatever>
    : tuple_element<Index, tuple<std::string, int, whatever>>
  {
  };
}

We even rely on tuple_element to do the bounds checking for us!

Step 4: Provide all of the get functions.

You have quite a few choices here. One option is to make the get functions members of your original class.

class Person
{
public:
  std::string name;
  int age;

  template<std::size_t Index>
  std::tuple_element_t<Index, Person>& get()
  {
    if constexpr (Index == 0) return name;
    if constexpr (Index == 1) return age;
  }
};

Or you can add them as free functions.

template<std::size_t Index>
std::tuple_element_t<Index, Person>& get(Person& person)
{
  if constexpr (Index == 0) return person.name;
  if constexpr (Index == 1) return person.age;
}

Adding them as free functions is convenient if you are trying to retrofit structured binding onto an existing class that you cannot change.

Another decision you have to make is which types of bindings you want to support, and what they will produce. The examples I gave above support mutable lvalue references and produce mutable lvalue references. This means that future assignments into the deconstructed values wll propagate into the original, assuming it too was captured by reference.

Person p;

auto&& [name, age] = p;
name = "Fred";
age = 42;

The auto&& captures p by universal reference, so the get calls are made on the original object p. Those gets return references, so the modifications to name and age are modifications into the original object p.

But here’s another case:

Person p;

auto [name, age] = p;
name = "Fred";
age = 42;

In this case, the auto captured p by value into the hidden variable. When the get calls are made on the hidden variable, they are made on a copy, which means that the modifications to name and age are modifications of the copy, not the original object p.

And then we have this:

const Person p;

auto&& [name, age] = p;

This fails to compile because the get methods do not support const Person. You probably want those to return const references to the name and age.

This means that in practice, you need const and non-const overloads of the get method. And while you’re at it, you may as well complete the set with the const and non-const rvalue overloads.

class Person
{
public:
  std::string name;
  int age;

  template<std::size_t Index>
  std::tuple_element_t<Index, Person>& get() &
  {
    if constexpr (Index == 0) return name;
    if constexpr (Index == 1) return age;
  }

  template<std::size_t Index>
  std::tuple_element_t<Index, Person> const& get() const&
  {
    if constexpr (Index == 0) return name;
    if constexpr (Index == 1) return age;
  }

  template<std::size_t Index>
  std::tuple_element_t<Index, Person>& get() &&
  {
    if constexpr (Index == 0) return std::move(name);
    if constexpr (Index == 1) return std::move(age);
  }

  template<std::size_t Index>
  std::tuple_element_t<Index, Person> const& get() const&&
  {
    if constexpr (Index == 0) return std::move(name);
    if constexpr (Index == 1) return std::move(age);
  }
};

Fortunately, you can consolidate a lot of this with a helper method that infers the necessary boilerplate.

class Person
{
public:
  std::string name;
  int age;

  template<std::size_t Index>
  auto&& get()       &  { return get_helper<Index>(*this); }

  template<std::size_t Index>
  auto&& get()       && { return get_helper<Index>(*this); }

  template<std::size_t Index>
  auto&& get() const &  { return get_helper<Index>(*this); }

  template<std::size_t Index>
  auto&& get() const && { return get_helper<Index>(*this); }

private:
  template<std::size_t Index, typename T>
  auto&& get_helper(T&& t)
  {
    static_assert(Index < 2,
      "Index out of bounds for Custom::Person");
    if constexpr (Index == 0) return std::forward<T>(t).name;
    if constexpr (Index == 1) return std::forward<T>(t).age;
  }
};

Note that we had to restore the static assertion because we are no longer relying on tuple_element to do the bounds checking.

It is more common to use free functions instead of member functions, in which case you would have something like this:

template<std::size_t Index, typename T>
auto&& Person_get_helper(T&& p)
{
  static_assert(Index < 2,
    "Index out of bounds for Custom::Person");
  if constexpr (Index == 0) return std::forward<T>(t).name;
  if constexpr (Index == 1) return std::forward<T>(t).age;
}

template<std::size_t Index>
auto&& get(Person& p)
{
  return Person_get_helper<Index>(p);
}

template<std::size_t Index>
auto&& get(Person const& p)
{
  return Person_get_helper<Index>(p);
}

template<std::size_t Index>
auto&& get(Person&& p)
{
  return Person_get_helper<Index>(std::move(p));
}

template<std::size_t Index>
auto&& get(Person const&& p)
{
  return Person_get_helper<Index>(move(p));
}

Now, there’s no requirement that the get methods return references. You can have them return values, and the structured binding will simply capture values rather than references. This is handy if the underlying object doesn’t have access to references.

class Person
{
public:
  std::string CalculateName() const;
  int CalculateAge() const;

  template<std::size_t Index>
  auto get() const
  {
    static_assert(Index < 2,
      "Index out of bounds for Custom::Person");
    if constexpr (Index == 0) return CalculateName();
    if constexpr (Index == 1) return CalculateAge();
  }
};

We’ll see an application of this trick next time.

Bonus chatter: Since the structured binding transformation is purely syntactic, there’s no rule that prevents you from having the get functions return things that are unrelated to the source of the binding. It’s probably not a great idea, though, since nobody will be expecting that.

The post How to add C++ structured binding support to your own types appeared first on The Old New Thing.

A brief introduction to C++ structured binding

https://devblogs.microsoft.com/oldnewthing/20201014-00/?p=104367

https://devblogs.microsoft.com/oldnewthing/?p=104367

C++17 introduced a feature known as structured binding. It allows a single source object to be taken apart:

std::pair<int, double> p{ 42, 0.0 };
auto [i, d] = p;
// int i = 42;
// double d = 0.0;

It seems that no two languages agree on what to call this feature. C# calls it deconstructing. JavaScript calls it destructuring. (Python doesn’t seem to have a specific name for this concept, although the common case where the source is a list does have the name list comprehension.) Python calls it unpacking. My guess is that C++ avoided both of these terms to avoid confusion with the word destructor.

There is a subtlety in the way structured binding works: Binding qualifiers on the auto apply to how the source is bound, not on how the destination is bound.¹

For example,

auto&& [i, d] = p;

becomes (approximately)¹

If p.get<N> exists If p.get<N> does not exist
auto&& hidden = p;
decltype(auto) i = p.get<0>();
decltype(auto) d = p.get<1>();
auto&& hidden = p;
decltype(auto) i = get<0>(p);
decltype(auto) d = get<1>(p);

where hidden is a hidden variable introduced by the compiler. The declarations of i and d are inferred from the get method or free function.²

(In a cruel twist of fate, if hidden is const or a const reference, then that const-ness propagates to the destinations. But the reference-ness of hidden does not propagate.)

The decltype(auto) means that the reference-ness of the return type is preserved rather than decayed. If get returns a reference, that reference or qualifier is preserved. This differs from auto which will decay references to copies.

All of this comes into play when you want to make your own objects available to structured binding, which we’ll look at next time.

Bonus chatter: There is no way to specify that you want only selected pieces. You must bind all the pieces.

¹ In reality, the bound variables have underlying type std::tuple_element_t<N, T> (where N is the zero-based index and T is the type of the source), possibly with references added. But in practice, these types match the return types of get, so it’s easier just to say that they come from get.

² My reading of the language specification is that the destination variables are always references:

[dcl.struct.bind]
3. … [E]ach vᵢ is a variable of type “reference to Tᵢ” initialized with the initializer, where the reference is an lvalue reference if the initializer is an lvalue and an rvalue reference otherwise.

and therefore the expansion of the structured binding would be

auto&& i = get<0>(p);
auto&& d = get<1>(p);

However, in practice, the compilers declare the destination variables as matching the return value of get, as I noted above.

So I must be reading the specification wrong.

(The text was revised for C++20, but even in the revision, it’s still a reference.)

¹ That’s because a structured binding really is a hidden variable plus a bunch of references to the pieces of that hidden variable. That’s why the qualifiers apply to the hidden variable, not to the aliases.

The post A brief introduction to C++ structured binding appeared first on The Old New Thing.