Front page | perl.perl6.language |
Postings from June 2010
Filesystems and files [Was: Re: The obligation of free stuff: GoogleStorage]
Thread Previous
|
Thread Next
From:
Richard Hainsworth
Date:
June 30, 2010 01:30
Subject:
Filesystems and files [Was: Re: The obligation of free stuff: GoogleStorage]
Message ID:
4C2B0062.7000300@rusrating.ru
If I might offer a late viewpoint after reading the Aaron's expanded
email (attached below).
When originally I suggested using 'open' instead of 'connect', the aim
was to keep consistency with the paradigm of files on the local system.
However, as Aaron's post suggests, when dealing with remote 'files',
there is an additional layer of functionality that must be introduced,
namely the need to 'connect' to the filesystem so that files on it can
be opened. Thus it is wrong to conflate 'open' with 'connect'.
It is normally implied that a program already has a 'local' environment,
including a 'local' filesystem. Thus the syntax
my $fn = open('/path/to/directory/filename', :r) or die $!;
implies a local file sytem.
The idea of an implied local system suggests an implied local
environment. The contents of %*ENV and @*INC seem to be assumed to be
local, thought this is not specified. Given the development of the
internet, this is an assumption I think should be made implicit, as well
as the mechanism for adding remote resources via paths through a network.
Would it make sense to define $*FS as the implied local file system, and
thus that a bare 'open' is sugar for
my $fh = $*FS.open('/path/to/directory/filename', :r);
This then means that there is an implicit
$*FS.connect();
that makes the local system available to the program.
I wonder whether this would also be a way of unifying the program
interface for difference Operating Systems, in that a program running on
a *nix machine would have $*FS of type IO::Filesystem::POSIX, while $*FS
for a Windows machine would have type IO::Filesystem::Windows, etc.
Then it would be possible, as Aaron has suggested to have
my $remote-fs = IO::Filesystem::Google.connect(%args);
my $fh = $remote-fs.open($path-or-url, :r);
and then treat $fh as specified elsewhere.
Morover, it would then be possible to do
$*FS = $remote-fs;
I would propose that this sort of flexibility would be useful for
programs that are embedded in other virtual environments, such as
browser plugins, or programs intended to run on thin clients that do not
have their own filesystems.
Another possibility would be to have
my $windows-from-linux = IO::Filesystem::Windows.connect(%args);
my $linux-system = $*FS;
$*FS = $windows-from-linux;
And then the files on a dual boot system can be accessed by the program.
On 06/21/10 02:35, Aaron Sherman wrote:
> First off, I again have to caution that this thread is conflating
> "open" with filesystem interaction. While open is one of many ways of
> interacting with a filesystem, it isn't even remotely sufficient (nor
> my immediate focus). One can ask for and modify filesystem metadata,
> security information, and so on as well as that for individual objects
> within the filesystem (which in the POSIX model is mostly files and
> directories). In a traditional POSIX/Unix model, programs (other than
> key OS utilities) don't usually do much with the structure of the
> filesystem. That's meant as an interactive task for an admin. However,
> in building a cloud-storage aware VFS-layer, managing the filesystem
> in terms of layout, allocation, security (access methods,
> authorization and authentication), payment models, and many other
> features are expected to be embodied in the access model. Just as an
> example, choosing and laying out what Amazon calls "buckets" is the
> equivalent of partitioning. That does need an interface.
>
> Now, we can just translate the Python bindings for Google Storage (and
> I believe there are already Perl 5 bindings for Amazon S3), but my
> inclination is to build a generic VFS that can handle POSIX-like
> filesystems as well as everything else from Windows/Mac specific
> features to full-blown cloud storage to more user-oriented storage
> options (Dropbox comes to mind).
>
> Every addressable storage model which could be treated as a filesystem
> should have a place in the Perl 6 VFS.
>
> Now, as to the question of overloading "open"... I'm not sure. I mean,
> it's pretty easy to say:
>
> URI.new($path).open(:ro)
>
> or
>
> open(URI.new($path), :ro)
>
> When what you want is a VFS object, and I kind of like the idea of the
> standard open on a string having POSIX semantics.
>
> Now, to your question, C.J.
>
> On Fri, Jun 18, 2010 at 3:03 PM, C.J. Adams-Collier
> <cjac@colliertech.org> wrote:
>
>> Define "opening a file" for me. Is it something that's associated with a
>> filehandle, by definition? Do TCP sockets count?
>>
> Opening a file isn't a well defined operation. You have to be more specific.
>
> In your question you're conflating the evaluation of a filesystem
> namespace token (which is one of many possible modes of filesystem
> interaction), returning a filehandle object that represents access to
> the named object with evaluation of a socket namespace token and
> returning a similar filehandle object that represents that object.
> There are, of course crossovers (filesystem pipes) and other
> operations that yield filehandle objects (various IPC operations that
> aren't exactly sockets, for example).
>
> Now, if you want to unify some of that territory, you can build a VFS
> layer. Today, that's often done via URIs, just because they're handy
> for Identifying Universal Resources, but what happens when you call
> open (or equivalent) on such a token is still an open question, and I
> didn't seek to answer it in this thread.
>
> Franky, I don't think that it's something that SHOULD be answered
> prior to building the VFS layer itself, because that layer might
> dictate some design decisions, but my high level impulse is to say
> that open on a VFS token (be it a URI or some othe complex data) will
> yield a VFS-back-end specific "handle". Such a handle would likely
> "do" IO::Handle and friends as well as something VFS-specific.
>
>
Thread Previous
|
Thread Next