Introduction

Data in the LTA is stored in one of the participating data centers, each of which is operating a dCache storage management environment. Within each data center, data is organized in "projects" for which the respective science teams can be granted temporary exclusive read and write access. A limited group of administrators is granted full access to all the data. The following simple directory structure is created in dCache:

<root-uri>/data/lofar/ops/projects/<project1>/...
<root-uri>/data/lofar/ops/projects/<project2>/...

The 'root-uri' is a unique path that includes the applicable protocol for data access and a data-center specific path root path for that protocol. This page documents how to access data using the WebDAV protocol. WebDAV is an extension of the HTTP protocol and the relevant data paths for the LOFAR LTA will start with https://.... The dCache WebDAV interface supports regular HTTP clients browsers, as well as command-line clients. There are advanced clients that are recommended for bulk operations. LOFAR specifically supports the clients that are documented on this page.

User access is provided by distributing tokens which grant authorizations in accordance with the applicable access model. Tokens are generated on the dCache systems and are of the 'macaroon' type. A given macaroon will only grant access to data on a single data center and may have further limitations. Users should read and follow the information that is provided through the LOFAR staging service to ensure that the appropriate macaroons are used for data access.

In case of questions of issues, please submit a support request to the ASTRON Science Data Center Helpdesk system (registration required), setting Component to 'SDC Services' and Ownergroup to 'SDCO'.

Warnings for working with the dCache WebDAV storage interface

⚠ Caution should be taken when using WebDAV clients that make the remote data archive look like a regular file system as these may trigger a large amount of requests that will slow down the server. Also, care should be taken to avoid (intended or accidental) transfers of huge amounts of data that can fill up available network bandwidth and/or local disk systems. Always check the total size of data (where applicable including content of sub-directories) before starting to copy it over to a local system