<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html lang="en" style="--code-editor-font: var(--default-mono-font, "GitLab Mono"), JetBrains Mono, Menlo, DejaVu Sans Mono, Liberation Mono, Consolas, Ubuntu Mono, Courier New, andale mono, lucida console, monospace;">
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<title>
GitLab
</title>
<style data-premailer="ignore" type="text/css">
a { color: #1068bf; }
</style>
<style>img {
max-width: 100%; height: auto;
}
body {
font-size: .875rem;
}
body {
-webkit-text-shadow: hsla(0,0%,100%,.01) 0 0 1px;
}
body {
font-family: "GitLab Sans",-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Noto Sans",Ubuntu,Cantarell,"Helvetica Neue",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji"; font-size: inherit;
}
</style>
</head>
<body style="font-size: inherit; -webkit-text-shadow: hsla(0,0%,100%,.01) 0 0 1px; font-family: "GitLab Sans",-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Noto Sans",Ubuntu,Cantarell,"Helvetica Neue",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";">
<div class="content">
<p style="color: #777777;">
<a href="https://gitlab.rtems.org/mohamedayman23">Mohamed Ayman</a>
<a href="https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/2129#note_158723">commented</a>:
</p>
<div class="md" style="position: relative; z-index: 1; color: #3a383f; word-wrap: break-word;">
<p dir="auto" style="color: #3a383f; margin: 0px 0px 1rem;" align="initial">Hi <a href="https://gitlab.rtems.org/sebhub" title="Sebastian Huber" class="gfm gfm-project_member js-user-link" data-user="28" data-original="@sebhub" data-container="body" data-placement="top" data-reference-type="user" style="color: #284779; background-color: #cbe2f9; border-radius: .25rem; margin-top: 0px; padding: 0 2px;">@sebhub</a> ,<a href="https://gitlab.rtems.org/gedare" title="Gedare Bloom" class="gfm gfm-project_member js-user-link" data-user="8" data-original="@gedare" data-container="body" data-placement="top" data-reference-type="user" style="color: #284779; background-color: #cbe2f9; border-radius: .25rem; padding: 0 2px;">@gedare</a> , <a href="https://gitlab.rtems.org/chris" title="Chris Johns" class="gfm gfm-project_member js-user-link" data-user="3" data-original="@chris" data-container="body" data-placement="top" data-reference-type="user" style="color: #284779; background-color: #cbe2f9; border-radius: .25rem; padding: 0 2px;">@chris</a> , <a href="https://gitlab.rtems.org/opticron" title="Kinsey Moore" class="gfm gfm-project_member js-user-link" data-user="4" data-original="@opticron" data-container="body" data-placement="top" data-reference-type="user" style="color: #284779; background-color: #cbe2f9; border-radius: .25rem; padding: 0 2px;">@opticron</a> , <a href="https://gitlab.rtems.org/joel" title="Joel Sherrill" class="gfm gfm-project_member js-user-link" data-user="7" data-original="@joel" data-container="body" data-placement="top" data-reference-type="user" style="color: #284779; background-color: #cbe2f9; border-radius: .25rem; padding: 0 2px;">@joel</a></p>
<p dir="auto" style="color: #3a383f; margin: 0px 0px 1rem;" align="initial">I've been digging into <a href="https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/2129" title="sync() implementation is dangerous" class="gfm gfm-issue" data-original="#2129" data-link="false" data-link-reference="false" data-issue="23243" data-project="26" data-iid="2129" data-namespace-path="rtems/rtos/rtems" data-project-path="rtems/rtos/rtems" data-issue-type="issue" data-container="body" data-placement="top" data-reference-type="issue" style="margin-top: 0px;">#2129</a> ("sync() implementation is dangerous") and wanted to share what I found, since it looks like the ticket may have been overtaken by events without anyone going back to close it out.</p>
<p dir="auto" style="color: #3a383f; margin: 0px 0px 1rem;" align="initial">For context on how I ended up here: the ticket describes three weaknesses in <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; margin-top: 0px; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> no protection against concurrent <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> calls, no Newlib locks meaning other tasks shouldn't touch file operations while <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> runs, and undefined behavior if a task is deleted mid-<code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code>. That's still exactly what the ticket says today, filed against 4.11/HEAD, milestone later moved to 5.0, and it's sat open ever since.</p>
<p dir="auto" style="color: #3a383f; margin: 0px 0px 1rem;" align="initial">What I found going through the git history of <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; margin-top: 0px; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">cpukit/libcsupport/src/sync.c</code>: back when this ticket was filed, <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> worked by walking each task's Newlib reentrancy structure and every <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">FILE*</code> stream open on it, calling <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">fsync()</code>/<code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">fdatasync()</code> on each one it found. That walk is exactly the mechanism the three weaknesses are about — nothing stopped a task from being deleted, or a file from being closed, while <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> was still walking it.</p>
<p dir="auto" style="color: #3a383f; margin: 0px 0px 1rem;" align="initial">That implementation doesn't exist anymore. In commit <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; margin-top: 0px; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">3ccfb58311</code> ("Synchronize all file descriptors in sync()", May 13 2022, your commit), <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> was rewritten from scratch to just loop over the fd table directly and call <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">fsync()</code>/<code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">fdatasync()</code> on each fd — no more per-task walk at all. The commit message says it closes <a href="https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/4656" title="sync() whould synchronize all file descriptors (cloned)" class="gfm gfm-issue" data-original="#4656" data-link="false" data-link-reference="false" data-issue="27224" data-project="26" data-iid="4656" data-namespace-path="rtems/rtos/rtems" data-project-path="rtems/rtos/rtems" data-issue-type="issue" data-container="body" data-placement="top" data-reference-type="issue">#4656 (closed)</a>, which as far as I can tell was about <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> missing files that weren't associated with a <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">FILE*</code> stream, not about the thread-safety issue here. So the fix for a completely different bug happened to remove the exact thing this ticket was worried about, as a side effect. I looked for a merge request tied to that commit and couldn't find one not sure if it went in as a direct commit or if I just couldn't locate the MR.</p>
<p dir="auto" style="color: #3a383f; margin: 0px 0px 1rem;" align="initial">I didn't want to just take that at face value, so I wrote a stress test to check it myself: two tasks hammering <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; margin-top: 0px; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> back to back, one task doing continuous <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">open()</code>/<code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">write()</code>/<code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">close()</code>, and one task continuously creating and exiting other tasks — all running in parallel on 4 real cores (<code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sis -leon3 -m 4</code>), against a real FAT filesystem mounted on a RAM disk (IMFS's <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">fsync_h</code> turned out to be a no-op, so I didn't want to test against that). 30 runs, 20,000 iterations per task, no crashes, hangs, or failures in any of them.</p>
<p dir="auto" style="color: #3a383f; margin: 0px 0px 1rem;" align="initial">So from where I'm sitting: the specific failure mode this ticket describes doesn't seem to exist in current <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; margin-top: 0px; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">main</code> anymore, but I want to be upfront about what I haven't checked I only tested FAT (not RFS/littlefs/JFFS2), and I only tested clean task exit, not a forced <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">rtems_task_delete()</code> mid-I/O, which seems like it could still be a real risk given there's no explicit lock serializing <code style="font-size: 90%; color: #18171d; word-wrap: break-word; background-color: #ececef; border-radius: .25rem; font-weight: inherit; overflow-wrap: break-word; white-space: break-spaces; font-family: "GitLab Mono", "JetBrains Mono", "Menlo", "DejaVu Sans Mono", "Liberation Mono", "Consolas", "Ubuntu Mono", "Courier New", "andale mono", "lucida console", monospace; font-variant-ligatures: none; word-break: keep-all; padding: 0.125rem 0.25rem;">sync()</code> even now.</p>
<p dir="auto" style="color: #3a383f; margin: 0px;" align="initial">Given you filed this originally and wrote the 2022 rewrite is this still something you consider open, or would you be comfortable closing it referencing that commit? And if there's a specific scenario you still had in mind that I'm not covering, I'm happy to extend the test and dig further..</p>
</div>
</div>
<div class="footer" style="margin-top: 10px;">
<p style="font-size: small; color: #626168;">
—
<br>
<a href="https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/2129#note_158723">View it on GitLab</a>.
<br>
You're receiving this email because of your account on <a target="_blank" rel="noopener noreferrer" href="https://gitlab.rtems.org">gitlab.rtems.org</a>. <a href="https://gitlab.rtems.org/-/sent_notifications/5-2gxer6bjhru83hgmi2dq2se5o-1d/unsubscribe" target="_blank" rel="noopener noreferrer">Unsubscribe</a> from this thread · <a href="https://gitlab.rtems.org/-/profile/notifications" target="_blank" rel="noopener noreferrer" class="mng-notif-link">Manage all notifications</a> · <a href="https://gitlab.rtems.org/help" target="_blank" rel="noopener noreferrer" class="help-link">Help</a>
<span style="color: transparent; font-size: 0; display: none; overflow: hidden; opacity: 0; width: 0; height: 0; max-width: 0; max-height: 0;">
Notification message regarding https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/2129#note_158723 at 1788520661
</span>
<script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","action":{"@type":"ViewAction","name":"View Work item","url":"https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/2129#note_158723"}}</script>
</p>
</div>
</body>
</html>