{"id":22,"date":"2015-02-24T03:36:55","date_gmt":"2015-02-24T03:36:55","guid":{"rendered":"http:\/\/www.msully.net\/blog\/?p=22"},"modified":"2015-02-25T04:05:24","modified_gmt":"2015-02-25T04:05:24","slug":"forcing-memory-barriers-on-other-cpus-with-mprotect2","status":"publish","type":"post","link":"https:\/\/www.msully.net\/blog\/2015\/02\/24\/forcing-memory-barriers-on-other-cpus-with-mprotect2\/","title":{"rendered":"Forcing memory barriers on other CPUs with mprotect(2)"},"content":{"rendered":"<p>I have something of an unfortunate fondness for indefensible hacks.<\/p>\n<p>Like I discussed in my last post, RCU is a synchronization mechanism that excels at protecting read mostly data. It is a particularly useful technique in operating system kernels because full control of the scheduler permits many fairly simple and very efficient implementations of RCU.<\/p>\n<p>In userspace, the situation is trickier, but still manageable. Mathieu Desnoyers and Paul E. McKenney have built a <a href=\"http:\/\/urcu.so\">Userspace RCU library<\/a> that contains a number of different implementations of userspace RCU. For reasons I won&#8217;t get into, efficient read side performance in userspace seems to depend on having a way for a writer to force all of the reader threads to issue a memory barrier. The URCU library has one version that does this using standard primitives: it sends signals to all other threads; in their signal handlers the other threads issue barriers and indicate so; the caller waits until every thread has done so. This is very heavyweight and inefficient because it requires running all of the threads in the process, even those that aren&#8217;t currently executing! Any thread that isn&#8217;t scheduled now has no reason to execute a barrier: it will execute one as part of getting rescheduled. Mathieu Desnoyers attempted to address this by adding a membarrier() system call to Linux that would force barriers in all other running threads in the process; after more than a dozen posted patches to LKML and a lot of back and forth, it got silently dropped.<\/p>\n<p>While pondering this dilemma I thought of another way to force other threads to issue a barrier: by modifying the page table in a way that would force an invalidation of the Translation Lookaside Buffer (TLB) that caches page table entries! This can be done pretty easily with mprotect or munmap.<\/p>\n<p>Full details in the <a href=\"https:\/\/github.com\/msullivan\/userspace-rcu\/commit\/04656b468d418efbc5d934ab07954eb8395a7ab0\">patch commit message.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I have something of an unfortunate fondness for indefensible hacks. Like I discussed in my last post, RCU is a synchronization mechanism that excels at protecting read mostly data. It is a particularly useful technique in operating system kernels because full control of the scheduler permits many fairly simple and very efficient implementations of RCU. [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[12,13,11,14],"class_list":["post-22","post","type-post","status-publish","format-standard","hentry","category-hacks","tag-hack","tag-not-research","tag-rcu","tag-weak-memory"],"_links":{"self":[{"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/posts\/22","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/comments?post=22"}],"version-history":[{"count":1,"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/posts\/22\/revisions"}],"predecessor-version":[{"id":23,"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/posts\/22\/revisions\/23"}],"wp:attachment":[{"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/media?parent=22"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/categories?post=22"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.msully.net\/blog\/wp-json\/wp\/v2\/tags?post=22"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}