CVS log for src/sys/fs/udf/udf_vnops.c

Up to [NetBSD] / src / sys / fs / udf

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase).  Plenty of mix'n match upper/lowercase has creeped
into the tree since then.  Nuke the macros and convert all callsites
to lowercase.

no functional change

Remove extra semicolons.

Apply patch (requested by reinoud in ticket #959):
UDF's flushing algorithm was susceptible to interference with genfs.
This patch fixes that and also ensures that there aren't tails around
being allocated later.

sync with head

Rework simplistic UBC flushing. Recent changes to genfs made the old flush
system generate heaps of odd allocations since the end of write request was
overwritten by the start of the second resulting in another relocation.

Also added a full flush of the file on a VOP_CLOSE(). This automatically
flushes file tails to disc.

sync with head.

Fix uid/gid check; it was checking if the old values would fit in 32 bits
instead of checking if the new values would.

System nodes are not written out on becomming inactive; they should be written
out before automatically.

However, when dealing with faulty discs that fail to mount, system nodes are
of course not written out and thus may still be marked dirty, if only due to
access. Especially on sequential media this gave rise to panics on reading
trackinfo since the write track section had not yet been initialised.

Sync with HEAD.

sync with head.

Apply patch (requested by reinoud in ticket #852):
Numerous UDF improvements:
- a much saner rmw backend
- free space management done right
- the high system time usage when encountering huge numbers of nodes
  are a thing of the past due to refactoring
- various small fixes

Since OSTA has not defined a good set of flags to cover our flags, its not
usefull to implement chflags(). Instead ignore it for now since the error
codes might confuse esp. gui's.

Replace the old hashtable and sorted list implemenation by a RB-tree.

Benefits are significant speed improvements on node creation/insertion while
keeping the lookup times low and still allowing sequential iteration over the
nodes.

Make source prettier

Pass the vp arg to udf_check_possible() so we can access its v_mount
member.

XXX No idea if this is the right solution to this problem, but it does
XXX at least allow thebuild to continue.  The original committed should
XXX verify that this does what was intended!

(Hello again, Elad)

Where possible, extract the file-system's access() routine to two internal
functions: the first checking if the operation is possible (regardless of
permissions), the second checking file-system permissions, ACLs, etc.

Mailing list reference:

	http://mail-index.netbsd.org/tech-kern/2009/06/21/msg005311.html

Rewrite of udf_on_rootpath(), and vop_rename() code that calls it, after the
UFS way. The tree walking is now done the same and the code hasn't locked up
on examples that made it lockup before.

Commit first stage of free-space accounting. It is estimating the underlimit
of free blocks on the device and when free blocks are getting tight it tries
to readjust/recalculate that value by syncing the FS.

Second stage will be resizing the data/metadata partitions.

Renaming in UDF was already possible but directories could only be renamed in
the same directory.

This patch finally allows a directory to be moved between parent directories.

Move the implementation of vaccess() to genfs_can_access(), in line with
the other routines of the same spirit.

Adjust file-system code to use it.

Keep vaccess() for KPI compatibility and to keep element of least
surprise. A "diagnostic" message warning that vaccess() is deprecated will
be printed when it's used (obviously, only in DIAGNOSTIC kernels).

No objections on tech-kern@:

	http://mail-index.netbsd.org/tech-kern/2009/06/21/msg005310.html

Add overlooked `cred' declaration in udf_write() when i committed the catchup
patches.

Pass-on credentials to size reversal on aborted write due to error. It could
be that in the future this credentials need to be changed to allways-allow
since its an reverting-on-error behaviour.

sync with head

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

Extract the open-coded authorization logic for chtimes() from various
file-systems and put it in a single function, genfs_can_chtimes().

This also makes UDF follow the same policy as all other file-systems.

Mailing list reference:

	http://mail-index.netbsd.org/tech-kern/2009/04/27/msg004951.html

sync with head.

Sync with HEAD.

Per discussion on tech-kern@:

  - Replace use of label/goto with returns

  - Rename, change prototype of, and move functions from vfs_subr.c to
    genfs_vnops.c

Refactor some duplicated file-system code.

Proposed and received no objections on tech-kern@:

	http://mail-index.netbsd.org/tech-kern/2009/04/18/msg004843.html

Pull up following revision(s) (requested by reinoud in ticket #592):
	sys/fs/udf/udf_vnops.c: revision 1.38
Fix panic due to memory leak on symlink creation. A test copy with 3000+
symlinks could get the machine down.

Fix panic due to memory leak on symlink creation. A test copy with 3000+
symlinks could get the machine down.

Apply patch (requested by reinoud in ticket #585):
Fix build by bringing in part of 1.35 that was left out.

Apply patch (requested by reinoud in ticket #569):
Remove dead code and double check if it's allowed to write out
non-sequential there.
--
Sequential writing goes before normal write. Its not likely to ever go
wrong but there could be a race otherwise.
--
Add simplistic UBC async flushing as other FS's seem to do. Its a clutch
and needs to be addressed in UBC properly.
Improve write throttling by doing it on page-size basis. The `standard'
solution would cut up files unnessisary during allocation. It also made
assumptions about ucb wich were propably right but still.

Remove all the __P() from sys (excluding sys/dist)
Diff checked with grep and MK1 eyeball.
i386 and amd64 GENERIC and sys still build.

Sync with HEAD.

Improve write throttling by doing it on page-size basis. The `standard'
solution would cut up files unnessisary during allocation. It also made
assumptions about ucb wich were propably right but still.

Add simplistic UBC async flushing as other FS's seem to do. Its a clutch and
needs to be addressed in UBC properly.

Sync with HEAD.

Sync with HEAD.

Pull up following revision(s) (requested by reinoud in ticket #192):
	sys/fs/udf/udf_vnops.c: revision 1.34
Fix endless loop occuring on empty directories listing on 512 bytes/sector
media like harddiscs. Also provide provide a `safety net' on readdir so it
will signal EOF when its requested an entry outside the directory.

Pull up following revision(s) (requested by reinoud in ticket #192):
	sys/fs/udf/udf_subr.c: revision 1.77
	sys/fs/udf/udf_vnops.c: revision 1.33
Elaborate SYNC debug messages and rename some debug messages to a more
proper category.

Fix endless loop occuring on empty directories listing on 512 bytes/sector
media like harddiscs. Also provide provide a `safety net' on readdir so it
will signal EOF when its requested an entry outside the directory.

Elaborate SYNC debug messages and rename some debug messages to a more
proper category.

Update haad-dm branch to haad-dm-base2.

Pull up following revision(s) (requested by reinoud in ticket #167):
	sys/fs/udf/udf_subr.c: revision 1.76
	sys/fs/udf/udf_vnops.c: revision 1.32
Split uid/gid remapping code to be independent of eachother.

Split uid/gid remapping code to be independent of eachother.

Rototill all remaining file systems to use ubc_uiomove() instead
of the ubc_alloc() - uiomove() - ubc_release() dance.

Sync with HEAD.

Sync with HEAD.

Sync with wrstuden-revivesa-base-2.

Sync with head.

Implement directory hashing to speed up directory traversals. Speed
improvements of at least 4 times in untarring and roughly 100 to 500 times
on file creation in big directories. Lookup of files was O(n*n) and is now
O(1) even for file creation. Free spaces in the directory are kept in a
seperate list for fast file creation.

The postmark benchmark gives:

UDF old:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
        1593 seconds total
        681 seconds of transactions (2 per second)

Files:
        3956 created (2 per second)
                Creation alone: 3000 files (4 per second)
                Mixed with transactions: 956 files (1 per second)
        990 read (1 per second)
        1010 appended (1 per second)
        3956 deleted (2 per second)
                Deletion alone: 2912 files (9 per second)
                Mixed with transactions: 1044 files (1 per second)

Data:
        5.26 megabytes read (3.38 kilobytes per second)
        21.93 megabytes written (14.10 kilobytes per second)
pm>



UDF new:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
        19 seconds total
        3 seconds of transactions (666 per second)

Files:
        3956 created (208 per second)
                Creation alone: 3000 files (230 per second)
                Mixed with transactions: 956 files (318 per second)
        990 read (330 per second)
        1010 appended (336 per second)
        3956 deleted (208 per second)
                Deletion alone: 2912 files (970 per second)
                Mixed with transactions: 1044 files (348 per second)

Data:
        5.26 megabytes read (283.66 kilobytes per second)
        21.93 megabytes written (1.15 megabytes per second)

Since the rename logic now locks its source directory too, we ought to
unlock the source directory again on exit. The stub that doesn't allow
cross directory renames for now jumped to the wrong exit point and thus
left a locked directory node that paniced on next locking.

Keep the directory to read '..' from locked until its dirent is read and
its location is extracted.

Also lock the source directory so its contents can safely be changed...
patch from tmpfs.

Fix chtimes permission checking. This sanitised version is hopefully DTRT.

Allow the renaming of directories *within* their current directory. No
cross-directory renaming is allowed yet.

Clarifying comments

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

Sync with HEAD.

Sync with head.

Make readsymlink bug-tolerant to MacOS X symlink construction bug. It puts
the name ".." on a parent path component. To prevent other similar errors,
name length checking is not done but the passed name that shouldn't be
passed is ignored.

We are not allowed zero length names in softlinks.

Support the creation of extended attributes for file creation in the
kernel and check and update them when found.

Note that it won't *insert* an extended attribute yet if it hasn't found it
while updating the birthtime.

Sync w/ -current. 34 merge conflicts to follow.

Cleanup file access,modification,attribute and birthtime setting of files
and make the modification time when set never older than the creation time.

sync with head

Sync with HEAD.

Implement wide char support for UDF by properly implementing unicode-16 to
UTF-8 like cd9660 and ntfs already do.

sync with head.

sync with head.

Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
        all operations can be performed on the file system though the
        sheduling is currently optimised for archiving workloads.

        mv(1)/rename(2) is currently only implemented for non-directories.

sync with HEAD

Sync with HEAD.

sync with head.

Remove VOP_LEASE. Discussed on tech-kern.

sync with head

Sync with HEAD

Fix dodgy tests of v_usecount.

sync with HEAD

Sync with HEAD

Merge vmlocking2 to head.

Sync with HEAD.

Sync with head.

Sync with HEAD

sync with head.

use __KERNEL_RCSID() instead of __RCSID()

Sync with HEAD.

sync with head

Pull the vmlocking changes into a new branch.

Sync with HEAD. amd64 Xen support needs testing.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start.  In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

sync with HEAD

sync with head.

Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.

sync with head.

Sync with head.

Merge from vmlocking: fix dodgy use of VOP_ISLOCKED().

sync with head.

Sync with HEAD.

Avoid stack allocation of large dirent structures in foo_readdir().

sync with head.

Fix dodgy use of VOP_ISLOCKED().

Sync with head.

Sync with head.

sync with head.

fix typos

Compile fixes.

sync with head.

Apply patch (requested by chs in ticket #422):
- Fix various deadlock problems with nullfs and unionfs.
- Speed up path lookups by upto 25%.

Sync with head.

Pull up following revision(s) (requested by reinoud in ticket #333):
	sys/fs/udf/udf_subr.c: revision 1.27-1.29
	sys/fs/udf/udf_vnops.c: revision 1.9
	sys/fs/udf/ecma167-udf.h: revision 1.6
Fix compilation issue when specifying different flags or compiling with a
different compiler. Thanks to Jared for noting.
Add UDF_ICB_FILETYPE_REALTIME type as found on UDF 2.50 discs. Also revert
a multiplication order to enforce 64 bit math.
Forgot one place for UDF_ICB_FILETYPE_RANDOMACCESS.

Add UDF_ICB_FILETYPE_REALTIME type as found on UDF 2.50 discs. Also revert
a multiplication order to enforce 64 bit math.

sync with head.

sync with head.

a smorgasbord of improvements to vnode locking and path lookup:
 - LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
   these now always return the parent vnode locked.  namei() works as before.
   lookup() and various other paths no longer acquire vnode locks in the
   wrong order via vrele().  fixes PR 32535.
   as a nice side effect, path lookup is also up to 25% faster.
 - the above allows us to get rid of PDIRUNLOCK.
 - also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
 - remove an assumption in layer_node_find() that all file systems implement
   a recursive VOP_LOCK() (unionfs doesn't).
 - require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
   fill in eopnotsupp() for file systems that don't support being exported
   and remove the checks for NULL.  (layerfs calls these without checking.)
 - in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
   adjust which vnode is locked.  fixes PR 33374.
 - apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().

Sync with head.

sync with head

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

sync with head

sync with head.

sync with head.

Sync with head.

remove ; from };

Sync with head.

Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.

sync with head.

integrate kauth.

sync with head

quell GCC 4.1 uninitialised variable warnings.

XXX: we should audit the tree for which old ones are no longer needed
after getting the older compilers out of the tree..

s/allready/already/

Adapt to kernel authorization KPI.

sync with head.

Fix $NetBSD$ tags. (sorry)

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.