Universal Disk Format

From Wikipedia, the free encyclopedia

Jump to: navigation, search

The Universal Disk Format (UDF) is a format specification of a file system for storing files on optical media. It is an implementation of the ISO/IEC 13346 standard (also known as ECMA-167). It is considered to be a replacement of ISO 9660, and as of 2009 is widely used for (re)writable optical media. UDF is developed and maintained by the Optical Storage Technology Association (OSTA).

Optical disc authoring
Optical media types
Discontinued Optical Disc Formats
Standards
Further reading
This box: view • talk • edit

Contents

[edit] History

The Optical Storage Technology Association standardized the UDF file system to form a common file system for all optical media. The goal was to make a common file system for read-only media and optical media that are re-writable. This is still the main goal for ongoing UDF standardization, although support for the more obscure WORM media is about to be limited[citation needed], and support for non-optical media may be added.

When first standardized, the UDF file system was intended to replace ISO 9660, allowing support for both read-only and writable media. Almost directly after the first version of UDF was released, it was adopted by the DVD Consortium as the official file system for DVD Video and DVD Audio. Nowadays, a UDF file system may be found on most authored optical discs in the market, and on almost all recordable DVD media that are used for video recording.

As intended, initially UDF operated mainly on optical media. Most operating systems needed special third-party software to support reading it. Nowadays, almost all operating systems natively support at least reading UDF file systems, and many support some form of writing as well. Because of this increased support, UDF is gaining popularity on non-optical media that mainly need to be exchangeable, such as Iomega REV discs, large flash media, and even on hard disk drives.

[edit] Media

DVD-Video media use UDF version 1.02. These discs contain a so-called UDF Bridge format, with both an ISO 9660 (Level 1) and a UDF 1.02 filesystem present on the same disc, and describing the same filesystem.

All standard formats for video recording on DVD-style media use some version of the UDF filesystem:

Normally, authoring software will master a UDF file system in a batch process and write it to optical media in a single pass. But when packet writing to rewriteable media, such as CD-RW, UDF allows files to be created, deleted and changed on-disc just as a general-purpose filesystem would on removable media like floppy disks and flash drives. This is also possible on write-once media, such as CD-R, but in that case the space occupied by the deleted files cannot be reclaimed (and instead becomes inaccessible).

[edit] Character set

The specification allows for nine character encodings: one by agreement, one specified by ECMA-6 (also known as ASCII), three subsets of ASCII, a subset of ECMA-94 (Latin-1), and various other graphical characters.

[edit] Revisions of UDF

Multiple revisions of UDF have appeared :

  • Revision 1.02 (August 30, 1996). This format is used by DVD-Video discs.
  • Revision 1.50 (February 4, 1997). Added support for (virtual) rewritability on CD-R/DVD-R media by introducing the VAT structure. Added sparing tables for defect management on rewritable media such as CD-RW, and DVD-RW and DVD+RW.
  • Revision 2.00 (April 3rd, 1998). Added support for Stream Files and real-time files (for DVD recording) and simplified directory management. VAT support was extended.
  • Revision 2.01 (March 15, 2000) is mainly a bugfix release to UDF 2.00. Many of the UDF standard's ambiguities were resolved in version 2.01.
  • Revision 2.50 (April 30, 2003). Added the Metadata Partition facilitating metadata clustering, easier crash recovery and optional duplication of file system information: All metadata like nodes and directory contents are written on a separate partition which can optionally be mirrored.
  • Revision 2.60 (March 1, 2005). Added Pseudo OverWrite method for drives supporting pseudo overwrite capability on sequentially recordable media.

For next releases of UDF, changes are discussed in relation to using UDF on very large hard disk media, and using UDF on holographic storage media.

[edit] Builds of UDF

While the UDF specification has never been explicit about it, all UDF revisions since 1.5 actually describe three different variations of the format.

These builds are:

  • Plain (Random Read/Write Access). This is the original format supported in all UDF revisions
  • VAT (Incremental Writing). Used specifically for writing to CD-R and (write-once) media
  • Spared (Limited Random Write Access). Used specifically for writing to CD-RW and DVD-RW (rewritable) media

[edit] Plain build

This format can be used on any type of disk that allows random read/write access, such as hard disks, DVD+RW and DVD-RAM media. Similarly to other common file system formats, such as FAT, directory entries point directly to the block numbers of their file contents. In writing to such a disk in this format, any physical block on the disk may be chosen for allocation of new or updated files.

Since this is the basic format, practically any OS or File System Driver claiming support for UDF should be able to read this format.

[edit] VAT build

DVD-R and CD-R media have limitations when being written to, in that each physical block can only be written to once, and the writing must happen incrementally. Thus the plain build of UDF can only be written to CD-Rs by pre-mastering the data and then writing all data in one piece to the media, similar to the way an ISO 9660 filesystem gets written to CD media.

To enable CD-R to be used virtually like a hard disk, whereby the user can add and modify files on a CD-R at will (so-called "drive letter access" on Windows), OSTA added the VAT build to the UDF standard.

The VAT is an additional structure on the disk that helps in remapping physical blocks when files or other data on the disk are modified. The write-once nature of the media means that when a file is first added and then deleted on the disk, the file's data still remains on the disk. It does not appear in the directory any more, but special tools can be used to access the previous state of the disc (the state before the delete occurred), making recovery possible. Eventually the disk will be full, as free space cannot be recovered by deleting files. However, this behavior can be used to advantage for the purpose of archiving data.

Understanding the VAT structure is necessary in order to read such discs, but not all UDF file systems support VAT. See also "Why your computer might not read a particular UDF disk", below.

[edit] Spared (RW) build

DVD-RW and CD-RW media have fewer limitations than DVD-R and CD-R media. Sectors can be rewritten at random (though in packets at a time). These media can be erased entirely at any time, making the disc blank again, ready for writing a new UDF or other file system (e.g. ISO 9660 or CD Audio) to it.

DVD-RW and CD-RW disks may thus be used as a blankable-R media but may also be formatted in the plain, VAT and Spared UDF builds.

However, it is important to understand that sectors of -RW media may "wear out" after a while, meaning that their data becomes unreliable, through having been rewritten too often (typically after a few hundred rewrites, with CD-RW).

If the plain build is used on a-RW media, file-system level modification of the data must not be allowed, as this would quickly wear out often-used sectors on the disc (such as those for directory and block allocation data), which would then go unnoticed and lead to data loss. Hence, if software formats -RW media with UDF in the plain build, it should set the "hard write protection" flag on the volume to ensure that no UDF software attempts to overwrite files on the volume as is possible with random-rewritable media such as hard disks.

To allow modification of files on the disc, the media can be used like -R media using the VAT build. This ensures that all blocks get written only once (successively), ensuring that there are no blocks that get rewritten more often than others. This way, a RW disc can be erased and reused many times before it should become unreliable.

To get true overwritability (which is not possible with the VAT build) of files on RW media, the disc needs to be formatted using the Spared build which adds an extra Sparing Table. This table keeps track of bad sectors and remaps them to working ones.

Once a -RW disc has been used with the spared UDF build, the disc should never be reused with any other format, as the information about the bad blocks would get lost, potentially leading to the aforementioned unreliability.

Since DVD+RW discs can't emulate DVD+R, they can only be formatted in the plain and in the Spared UDF build.

[edit] Consequences of using specific builds

The consequences of using these builds are as follows:

  • When using the plain build, in theory a disc driver may allow rewriting any disc sector at random, meaning the RW is truly rewritable in the fashion of hard disks. However, because of the wear-out effect, this would soon lead to loss of data. For that reason, if a plain UDF file system is written to RW media, the file system should lock (write protect) the UDF volume to prevent accidental modification by a computer, or better, disc drivers should never even attempt to provide random-write access to RW media unless they can assure that no data loss due to wear-out can happen.
  • When using the VAT build, CD-RW/DVD-RW media effectively appears as CD-R or DVD+/-R media to the computer. However, the media may be erased again at any time.
  • Finally, the Spared build works basically like the plain build, but uses an extra Sparing Table to remap worn-out sectors.

Understanding Sparing Tables is necessary to be able to read discs written in Sparable build correctly. The problem is that some existing versions of UDF File System software ignore this extra information and treat such UDF discs as if they had the plain build. As long as the media has no worn-out sectors, this does not matter - the files can be read properly. But once sectors are remapped, a File System not paying attention to the Sparing Table will read outdated sectors, leading to retrieval of the wrong data.

[edit] Why a computer might not read a particular UDF disk

Even if an operating system claims to be able to read UDF 1.50, it still may only support the plain build and not necessarily either the VAT or Spared build.

An example is Mac OS X (10.4.5), which claims to support UDF 1.50 (see man mount_udf), yet it can only mount disks of the plain build properly (it cannot mount UDF disks with a VAT at all, see Sony Mavica problem, and while it appears to be able to mount CD-RWs written with a Sparing Table, it does not read its files correctly in the case those files are actually remapped).

The solution is to update the computer’s operating system (OS) to any later version of the OS, available without charge up to version 10.4.11 for both Intel and PowerPC Macintosh computers.

Similarly, Microsoft Windows XP Service Pack 2 (SP2), cannot read DVD-RW disks that use the Universal Disk Format (UDF) 2.00 defect management system. When you view the DVD contents by using Windows Explorer, you see an empty root folder. This problem occurs if the UDF defect management system creates a sparing table that spans more than one sector on the DVD-RW disk. Windows XP SP2 cannot read a sparing table that is larger than one sector. Windows XP SP2 can recognize that a DVD is using UDF. However, Windows Explorer displays the contents of a DVD as an empty folder.

One can update the computer to Windows XP Service Pack 3, or to apply a hotfix to address this specific problem.

For solutions to more general problems reading UDF-formatted optical disks in computers using Microsoft Windows OSes, see Microsoft Windows UDF Read Troubleshooting.

[edit] Standalone DVD player compatibility

Some users have reported that video DVDs burned on their computers in UDF version 1.5 are not compatible with their set-top video DVD players. These players seem to be only compatible with UDF version 1.02 (that provides both UDF and ISO 9660 directories.) As some popular DVD burning programs default to UDF version 1.5 when burning video DVDs, users have found it necessary to avoid using the software wizard and instead manually configure the burn to version 1.02.

[edit] Verifier Software

Philips provides source code and binaries for a UDF verifier that examines media for UDF compliance.[1]

[edit] References

  • ISO/IEC 13346 standard, also known as ECMA-167.
  1. ^ "UDF Verification Software". Philips. 2007-11-25. http://www.ip.philips.com/services/?module=IpsVisitorLogin. Retrieved on 2007-11-25. 

[edit] Native OS support

Terminology:

  • Unless otherwise noted, read and write support means that only the plain UDF build is supported, but not the VAT and spared build.
  • Support for read means that a UDF formatted disk can be mounted by the system. It enables the user to read files from the UDF volume using the same interface that is used to access files on other disks connected to the computer.
  • Support for write means that, additionally to reading files from a mounted UDF volume, data such as files can also be modified, added, or deleted.
UDF version Non-plain
Operating system 1.02 1.50 2.0x 2.50 2.60 VAT Spared Write Note
AIX 5.1, 5.2, 5.3 Yes
AmigaOS 4.0 Yes Yes
eComStation Yes Yes
FreeBSD 5 Yes Yes No Yes
FreeBSD 6 Yes Yes No Yes
FreeBSD 7 Yes Yes No Yes
Linux 2.4 Yes Yes
Linux 2.6 Yes Yes Yes Yes (2.6.26+) Yes Version before 2.6.10 supported fewer media types.
Mac OS 9 Yes Yes Yes
Mac OS X 10.4 Yes Yes Yes Can create UDF 1.50 (plain build) volumes using the drutil utility.
Mac OS X 10.5 Yes Yes Yes Yes Yes Yes To create, use "newfs_udf" utility.
magnussoft ZETA 1.2.1 Yes Yes Yes Yes Yes Yes
NetBSD 4.0 Yes Yes Yes Yes Yes Yes Yes Reading multi-session VAT, spared and metapartition variants
from all CD, DVD and BD variants as well as HDD and Flash media.
NetBSD 5.0 Yes Yes Yes Yes Yes Yes Yes Yes Write support for all builds and media including multi-session VAT. Create new with "newfs_udf".
Limited writing on 2.50/2.60 (due to needing pre-allocated, fixed sized metadata partition).
Novell NetWare 5.1
Novell NetWare 6
OpenBSD 3.8 Yes Yes No Yes
OS/2 Yes Yes Additional fee drivers.
Solaris 7 11/99+ Yes Yes
Solaris 8, 9, 10 Yes Yes Yes
Windows 98/Me Yes Yes No No No No
Windows 2000 Yes Yes No No No No
Windows XP/Server 2003 Yes Yes Yes No No No Write support available with third party utilities such as DLA and InCD.
Windows Vista Yes Yes Yes Yes Yes Yes Yes Yes Referred to by Microsoft as Live File System.
Windows 7 Yes Yes Yes Yes Yes Yes

[edit] See also

[edit] External links

v • d • e
Standards of Ecma International
ANSI escape code · C++/CLI · C# · CD-ROM · CLI · DDS · E4X · ECMAScript · Eiffel · FAT · FD · HVD · Office Open XML · U3D · UDF · UDO · UMD · UWB
List of Ecma standards
Retrieved from "http://en.wikipedia.org/wiki/Universal_Disk_Format"
Personal tools