How to convert "High Efficiency" Nikon files?

Citizen of Bath · Monday at 5:24 PM

On my Nikon Z 9, I always use the Nef format, but, yesterday, my camera switched somehow to a format called "High Efficiency*", which is apparently a fake raw format (if it's lossy, it cannot be called raw). And now all of today's pictures are in this format.

How can I convert these files into Nef (the format called "Lossless compression" in the menu)? Apparently, NX Studio doesn't do that and I can't find any information on Nikon's websites.

That "High Efficiency*" thing really is a rotten file format. I can't even see the images on the Finder, using the current version of MacOS (15.2).

stickytape · Monday at 5:49 PM

Don't think it's a aRAW format, it's one of those annoying formats that is default on my iPhone unless I shoot RAW. Annoying though, for sure!

GetBusyLiving · Monday at 5:55 PM

It seems to me, in this age of massive storage devices and fast computers, that using any compression at all is pointless. I only ever shoot RAW and only ever will; the only time I take a picture in JPG is if I'm going to send a picture of a document via email and don't want to have to convert it.

Box Brownie · Monday at 6:07 PM

I found this IIRC by @sk66

View: https://youtu.be/RLLn3nG6uM8?si=A9ZHWWP6KZhbBEdy

In case it helps?

None too sure about conversion but the most recent version of FastRawViewer can display the actual HE file again to the way FRV truly displays raw files rather than the embedded JPEGs

FastRawViewer 2.0.9 Release

Camera/RAW format support Nikon HE/HE* compression support (for Nikon Z8, Z9, Zf, Z6_3 cameras) OM System 14-bit files (OM-1 Mark II HighRes/PixelShift shots) 4-channel/bayer JPEG-compressed DNG files Camera support Canon EOS R5 Mark II, EOS R1 Fujifilm X-T50, GFX 100S II Leica D-Lux8, Q3 43...

www.fastrawviewer.com

Peter123 · Monday at 6:53 PM

HE and HE* are both raw but compressed meaning smaller files. It is still a NEF file.
I always shoot this on my Z8 and develop without any problems in lightroom.
Doubt if you could tell any difference between this and lossless.

sk66 · Monday at 6:53 PM

Citizen of Bath said:
On my Nikon Z 9, I always use the Nef format, but, yesterday, my camera switched somehow to a format called "High Efficiency*", which is apparently a fake raw format (if it's lossy, it cannot be called raw). And now all of today's pictures are in this format.

How can I convert these files into Nef (the format called "Lossless compression" in the menu)? Apparently, NX Studio doesn't do that and I can't find any information on Nikon's websites.

That "High Efficiency*" thing really is a rotten file format. I can't even see the images on the Finder, using the current version of MacOS (15.2).

You cannot convert the Tico Raw (HE) formats back into lossless compressed.

It also helps to understand what the "lossy" bit means... The He* format is lossy in that the file cannot be uncompressed to the exact same original byte structure as a lossless compressed raw can; but in terms of usable bytes/data, there is no real loss. The higher compressed He format is probably better equated to recording an uncompressed raw file at 12bit vs 14bit like some older Nikons allowed. Tico states that it is visually lossless down to 1 bit per pixel (so 13bit instead of 14). I couldn't find any difference; but there may be some potential difference/loss in some particular/odd situation... maybe I don't have anything capable of showing/displaying such a difference (software processing, monitor, etc).

I'm personally using the highest compression He format; I have been since I ran those tests and made that video, and I have still found no issues. But if you want to return to recording lossless compressed you need to change the "RAW Recording" setting in the Photo Shooting Menu.

There is an issue with Mac's not showing thumbnails (reading the imbedded jpegs) for the He raw files.

Citizen of Bath · Monday at 6:53 PM

Box Brownie said:
I found this IIRC by @sk66

View: https://youtu.be/RLLn3nG6uM8?si=A9ZHWWP6KZhbBEdy

In case it helps?

None too sure about conversion but the most recent version of FastRawViewer can display the actual HE file again to the way FRV truly displays raw files rather than the embedded JPEGs

FastRawViewer 2.0.9 Release

Camera/RAW format support Nikon HE/HE* compression support (for Nikon Z8, Z9, Zf, Z6_3 cameras) OM System 14-bit files (OM-1 Mark II HighRes/PixelShift shots) 4-channel/bayer JPEG-compressed DNG files Camera support Canon EOS R5 Mark II, EOS R1 Fujifilm X-T50, GFX 100S II Leica D-Lux8, Q3 43...

www.fastrawviewer.com

Thank you very much, lots of information in there! Apparently those "High Efficiency*" files are almost raw and I'll have to make do with today's pictures in that format. At least, FastRawViewer solves the Finder issue.

Citizen of Bath · Monday at 6:59 PM

sk66 said:
You cannot convert the Tico Raw (HE) formats back into lossless compressed.

It also helps to understand what the "lossy" bit means... The He* format is lossy in that the file cannot be uncompressed to the exact same bytes as a lossless compressed raw can; but in terms of usable bytes/data, there is no real loss. The higher compressed He format is probably better equated to recording an uncompressed raw file at 12bit vs 14bit like some older Nikons allowed. Tico states that it is visually lossless down to 1 bit per pixel. I couldn't find any difference; but there may be some potential difference/loss in some particular/odd situation... maybe I don't have anything capable of showing/displaying such a difference (software processing, monitor, etc).

I'm personally using the highest compression He format; I have been since I ran those tests and made that video and I have still found no issues. But if you want to return to recording lossless compressed you need to change the "RAW Recording" setting in the Photo Shooting Menu.

There is an issue with Mac's not showing thumbnails (reading the imbedded jpegs) for the He raw files.

Great video, congratulations!

Box Brownie · Monday at 7:03 PM

Citizen of Bath said:
Thank you very much, lots of information in there! Apparently those "High Efficiency*" files are almost raw and I'll have to make do with today's pictures in that format. At least, FastRawViewer solves the Finder issue.

Also AFAIK of note.....

As FRV now "sees" them by default(?) LibRaw which is used by the likes of Affinity Photo means that more systems & PP programs will be able to process them too.

sk66 · Monday at 7:11 PM

GetBusyLiving said:
It seems to me, in this age of massive storage devices and fast computers, that using any compression at all is pointless. I only ever shoot RAW and only ever will; the only time I take a picture in JPG is if I'm going to send a picture of a document via email and don't want to have to convert it.

There is a huge difference between "compression" and jpeg processing... while a jpeg is at 8bit and compressed; the main issue is the lossy editing applied to the data in the process. But there is a whole lot of (essentially) useless data in a 14bit raw file that could be compressed/discarded.

All of that data (bit depth) becomes redundant when it is recording useless data. One example would be when the scene contains fewer stops of dynamic range than the sensor is capable of. And this is typical of when you use an ISO higher than base... i.e. a higher ISO is used because the scene does not contain the maximum brightness recordable. In this case the excess DR capability (bit depth) is only recording noise; because the minimum sensitivity is invariable and the minimum recordable light level cannot be reduced/changed.

It also becomes less useful at higher/brighter exposure values.

The difference between 0 and 2 bits is one stop (0-1); and the difference between 8192 and 16,384 bits is also one stop (13-14). And in terms of human perception those stops are equivalent; because human perception is also logarithmic.

The Weber ratio is 0.14 for the cone cells in the human eye (daytime color vision). That means the intensity must change at least 14% from where it was before in order to be perceivable by a human. And that only requires ~ 7 values measured/recorded within each stop.

So what happens is that you are effectively measuring/recording the linear DR (e.g. 14 stops in linear 1 stop increments) with a logarithmic scale which has increments much finer than a human can perceive... i.e. that last stop of DR only requires 7 values, but it is being recorded with 8,192 values instead. In this case the excess isn't noise, it just isn't useful... and this is where a lot of "visually lossless" compression can be achieved.

It's like recording all of the numbers on the tape measure when measuring to the nearest meter...

sk66 · Monday at 7:12 PM

Citizen of Bath said:
Great video, congratulations!

Thanks!