Recovery files overrunning disk space -

Recovery files overrunning disk space

Posted 2024年3月24日 GMT+1 01:53Cluster & Cloud Computing, Results & VisualizationVersion 6.02 Replies

Nicholas Pomata

Please login with a confirmed email address before reporting spam

I've helping a colleague run some COMSOL jobs (multiphysics simulations involving a parametric sweep) on our university's computing cluster. These require several terabytes of RAM and/or temporary storage space, which we have access to on the compute nodes as the job is running; however we do not have quite the same degree of storage available while the jobs are not actively running. This is a problem when COMSOL tries to write 1TB+ of recovery files to a filesystem that is less than 1TB away from its group quota.

The observed ensuing behavior is that COMSOL exits due to exceeded disk quota; then the next run seems to see that the recovery files are not intact (hardly a surprise if the last attempt failed while trying to write them) and starts over.

For further context (as much as I can give without currently having access to the recovery files) one attempt produced hundreds of savepoint subdirectories in addition to about 50k.mph.binfiles; I believe the latter is what took up most of the space (perhaps relevantly, this is on a striped filesystem with a large block size).

My questions, then, are:

What options can we invoke to reduce the amount of space COMSOL uses for recovery, short of simply not saving recovery files at all?
- On the flipside, should we expect that the final output file would actually be roughly that big?
- Can we get COMSOL to use fewer, large files instead of saving so many small files?
- Are there any files in this recovery folder that can be safely deleted after a certain point? How would these be identified?
- If we were to run a process that (say) ran behind COMSOL zipping up recovery files once they were written, would this interfere with its operations?
Failing that, is it possible to impose a limit on the amount of disk space COMSOL will use so that it doesn't interfere with other users from our group who rely on the same scratch space?
As for the existing recovery files, are there shell commands to convert them to a combined (hopefully smaller).mphfile or export their contents?
- What about only parsing some of them (especially if others are corrupted)?

Thanks for any help.

Reply

2 Replies Last Post 2024年3月24日 GMT+1 16:47

Robert Koslover Certified Consultant

Send Private Message Flag post as spam

Please login with a confirmed email address before reporting spam

Posted: 5 months ago

Updated: 5 months ago

I don't think you have to save recovery files. Go to File, Preferences, Files, Recovery, and uncheck "Keep recovery files when the application terminates unexpectedly." You can also change the path and folder for recovery files. Search for "recovery file" in the Help system for more info. There's a section there called "Keeping and Opening Recovery Files."

-------------------
Scientific Applications & Research Associates (SARA) Inc.
www.comsol.com/partners-consultants/certified-consultants/sara

Nicholas Pomata

Send Private Message Flag post as spam

Please login with a confirmed email address before reporting spam

Posted: 5 months ago

Updated: 5 months ago

I don't think you have to save recovery files. Go to File, Preferences, Files, Recovery, and uncheck "Keep recovery files when the application terminates unexpectedly." You can also change the path and folder for recovery files. Search for "recovery file" in the Help system for more info. There's a section there called "Keeping and Opening Recovery Files."

Thanks for the reply, but to be clear I would like to still keep recovery files; each failed attempt takes a decent chunk of our group's limited compute resources and I would prefer not to have to restart from scratch. (Additionally we learned how to change the tmp & recovery directories after the tmp files overran the much smaller home directory.)

Reply

Please read thediscussion forum rulesbefore posting.

Discussion Forum Rules and Guidelines

The goal of COMSOL Access is to provide a forum for you to communicate effectively with COMSOL as well as your colleagues within the multiphysics simulation community. This involves providing you with access to technical support and downloads of the latest {:comsol} software releases, as well as the ability to share your comments and work with other users of the {:comsol} software through forums such as the blog, discussion forum, and Application Exchange. In order to make this an efficient and pleasant experience for you and other members of COMSOL Access, we ask that you follow a few rules and guidelines.

When registering for COMSOL Access, you agree to provide your complete and truthful information for all fields requested on your COMSOL Access account registration page. You also agree to maintain the accuracy of all information associated with you on your COMSOL Access account. You agree to maintain your COMSOL Access account for use solely by you, not to share your username and password with anyone else, and to take appropriate precautions to restrict access to your username and password from others. Furthermore, you agree not to submit any information relating to your employer through your COMSOL Access account without your employer’s authorization. Should you use a COMSOL Access account associated with an employer, you agree to immediately discontinue using that account upon termination of that employment.

The moderators of the forums will remove any generally objectionable material as quickly as possible. You acknowledge that all posts made to these forums express the views and opinions of the author and not the administrators, moderators, or webmaster (except for posts by these people). Hence, they will not be held liable.

You agree not to post or link to any material that is abusive, obscene, vulgar, slanderous, hateful, threatening, sexually oriented, or that infringes upon or violates any third-party rights or any other material that may violate any applicable laws. You agree that you will not otherwise use your COMSOL Access account to violate or to assist anyone in violating any law. Engaging in any activity in violation of these COMSOL Access rules and guidelines may lead to you being immediately and permanently banned from COMSOL Access. The IP address of all posts is recorded to aid in enforcing these conditions. You agree that the webmaster, administrator, and moderators of the forums have the right to remove, move, or close any topic at any time as they see fit. As a user of COMSOL Access, you agree to any information you have entered into any of the forums being stored in a database.

You agree that you will not use your COMSOL Access account in violation of any applicable export control laws. You represent and warrant that you are not subject to any comprehensive sanction or embargo by the U.S. or any other country, nor are you identified on any list maintained by the U.S. government that identifies persons for which the U.S. maintains restrictions. Further, you represent and warrant that you are not subject to any restriction on the receipt of technology or products under the export control laws of the U.S. or any other country.

Basic Rules

Flaming:Do not post any messages that harass, insult, belittle, threaten, or flame another member or guest. Debates are fine, but argue with the point, not the person.
Trolling:Do not post with the purpose of starting a dispute. Note that a person disagreeing with your opinion is not considered trolling; keep it civil, even if you are sure that the other person is wrong. Anything seen as trolling will result in you being banned.
Spamming:Posts without content or containing nonsense waste space and everyone's time. Spam will be removed from the forum.
Offensive Posts, Links or Images:Do not use profanity, racial, ethnic, religious, or other slurs or any other offensive material.
Advertising:Posting of advertisements for products or services, links to auctions, affiliate links, links to promote websites, and so forth is not allowed.
Disclosure of Personal Information:Do not disclose any other member's email, real name, address, phone number, IP address, or other personal information. This includes posting contents of emails and private messages without the sender's consent. Do not bring personal disputes into the forums. Bans and warnings issued to other users are considered personal information.
Misleading Titles:The subject line of the post should be as informative as possible about the content of the post. This also reduces the amount of duplicate posts.
Thread Hijacking:Taking a thread off-topic to pursue one's own agenda is not permitted.
Piracy:Do not upload, post, email, transmit, or otherwise make available any content that infringes upon any patent, trademark, trade secret, copyright, or other proprietary rights ("Rights") of any party. This includes COMSOL model files created by a forged or illegitimate license.
Honesty:Users must provide truthful information in creating their COMSOL Access account.

These rules are subject to change. The moderators reserve the right to remove, edit, or move posts at their discretion. The COMSOL Access administrators will reserve the right to permanently remove a user account without notice if any of the rules are not followed. Particular services accessible with your COMSOL Access account may be subject to additional rules. You agree to comply with all rules applicable to each service you access through your COMSOL Access account.

Posting Guidelines

When posting, understand that you are trying to communicate with other people. Although many COMSOL Access members are not fluent in English, the official language of this forum is English.

Here are some important guidelines of language:

Write in English.
If you are familiar with LaTeX, please use this to write mathematical equations.
Always do a quick check for spelling/grammar mistakes.
Format your post in a legible manner. Use the Preview button often.
Be concise and articulate as much as possible.
Use the Enter key to create paragraphs.
Capitalize correctly. It is difficult to read posts that are written entirely in uppercase or lowercase.
Use correct punctuation. Avoid run-on sentences.
Try to avoid using “text speak”, “net speak”, or slang. The purpose of language is to be understood.
Never invent acronyms and use as few acronyms as possible. For example, write "COMSOL Multiphysics" and not "CMP".
Review your post before publishing it. Try to catch typos.
Please check to see if a topic has already been posted. Do not post multiple threads on the same topic.

Disclaimer

By submitting content to the forums, you hereby grant COMSOL a nonexclusive, royalty-free, perpetual, worldwide, and unrestricted license to reproduce, publicly display, publicly distribute, and prepare derivative works of the content. COMSOL hereby grants you a license to copy and/or use content from the forums solely for your own internal purposes. COMSOL provides the forum service for the benefit of our users to share content with the community. All content is provided "as is" without warranty of any kind, express or implied, including without limitation, warranties of merchantability, noninfringement, design, operation, and fitness for a particular purpose, and the entire risk as to the quality and performance of the programs is with you.

Neither COMSOL, the authors, nor the copyright owners of submitted materials warrant that the programs will be error-free, uninterrupted, virus-free, secure, and suitable for your needs, produce specific results, or that errors or failures will be corrected. Comments on supplied content should be sent to the author or copyright owner through the tools provided in the forums.

Pleaselog into post a reply.

Discussion Forum