MERRITT-L Archives

Merritt users list

MERRITT-L@LISTSERV.UCOP.EDU

Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Perry Willett <[log in to unmask]>
Reply To:
Merritt users list <[log in to unmask]>
Date:
Wed, 1 Apr 2015 20:28:47 +0000
Content-Type:
text/plain
Parts/Attachments:
text/plain (21 lines)
We're currently experiencing a very high load on Merritt. We're very close to finishing replicating content from SDSC to UCLA-just the largest objects are left. But one outcome of replication is that the size of our audit process has doubled-we're now running the audit process in both SDSC and UCLA. This doubles the load on the database. We're looking into how to optimize our database use for this increased load, but in the meantime the response is sluggish.

In addition, we've received a very large submission-over 120,000 objects. We're processing these new objects, although with the increased database load described above, it's going more slowly than we'd like. This has a couple of impacts:

1)      Please don't submit objects larger than 2 GB. The amount of free space in the staging area on our ingest servers is down to 30 GB (normally each server has 250 GB). As we process the large submission, the amount of free space will increase, but if you have objects larger than about 2 GB, please wait until next week to submit them. It's fine to continue submitting objects smaller than 2GB.

2)      Inventory updates may take a day or longer. We have a couple of queues as part of the ingest submission process. The first one is for the initial request. It's pretty full, but when we receive new submissions, we'll promote them to the front of the queue while we continue to process other large submissions. The second one is for adding information to the database (that we call "inventory.") Given the situation described above, we're processing these slowly. We can also promote inventory updates to the top of the queue if needed, but otherwise, all requests are added to the bottom of the queue. It may take a day or two after you submit content for it to show up in the UI.

We apologize for this temporary situation-it should be back to normal by next week.

It's clear that we need to scale up Merritt-objects are getting larger, in both the number and size of files being submitted. We'll need to find ways to optimize and increase capacity for submission, retrieval and other processes in response. Please let me know if you have any questions or concerns, particularly about any the status of any content you may have already submitted or are planning to submit. Thanks,

Perry Willett
Digital Preservation Services Manager
California Digital Library
415 20th St., 4th Floor
Oakland CA 94612-2901
Ph: 510-987-0078
Fax: 510-893-5212
Email: [log in to unmask]<mailto:[log in to unmask]>

ATOM RSS1 RSS2