Вопрос по posix, c, file-io – Атомность `write (2)` к локальной файловой системе

16

Error: User Rate Limit Exceeded

Either a file descriptor or a stream is called a "handle" on the open file description to which it refers; an open file description may have several handles. […] All activity by the application affecting the file offset on the first handle shall be suspended until it again becomes the active file handle. […] The handles need not be in the same process for these rules to apply. -- POSIX.1-2008

Error: User Rate Limit Exceeded

If two threads each call [the write() function], each call shall either see all of the specified effects of the other call, or none of them. -- POSIX.1-2008

Error: User Rate Limit Exceededwrite(handle, data1, size1)Error: User Rate Limit Exceededwrite(handle, data2, size2)Error: User Rate Limit Exceededdata1Error: User Rate Limit Exceededdata2 mustError: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/wait.h>
die(char *s)
{
  perror(s);
  abort();
}

main()
{
  unsigned char buffer[3];
  char *filename = "/tmp/atomic-write.log";
  int fd, i, j;
  pid_t pid;
  unlink(filename);
  /* XXX Adding O_APPEND to the flags cures it. Why? */
  fd = open(filename, O_CREAT|O_WRONLY/*|O_APPEND*/, 0644);
  if (fd < 0)
    die("open failed");
  for (i = 0; i < 10; i++) {
    pid = fork();
    if (pid < 0)
      die("fork failed");
    else if (! pid) {
      j = 3 + i % (sizeof(buffer) - 2);
      memset(buffer, i % 26 + 'A', sizeof(buffer));
      buffer[0] = '-';
      buffer[j - 1] = '\n';
      for (i = 0; i < 1000; i++)
        if (write(fd, buffer, j) != j)
          die("write failed");
      exit(0);
    }
  }
  while (wait(NULL) != -1)
    /* NOOP */;
  exit(0);
}

Error: User Rate Limit Exceededgrep -a '^[^-]\|^..*-' /tmp/atomic-write.logError: User Rate Limit Exceeded

Error: User Rate Limit ExceededO_APPENDError: User Rate Limit Exceededopen(2)Error: User Rate Limit Exceeded

O_APPEND If set, the file offset shall be set to the end of the file prior to each write.

Error: User Rate Limit Exceededlseek(2)Error: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

Error: User Rate Limit ExceededError: User Rate Limit Exceededfile descriptionError: User Rate Limit Exceeded

Error: User Rate Limit ExceededError: User Rate Limit Exceeded

The write call as defined in POSIX has no atomicity guarantee at all.

Error: User Rate Limit ExceededError: User Rate Limit ExceededError: User Rate Limit ExceededO_APPENDError: User Rate Limit ExceededO_APPEND.

Error: User Rate Limit Exceeded

Error: User Rate Limit Exceeded David Schwartz
Error: User Rate Limit Exceededimages.apple.com/macosx/docs/OSX_for_UNIX_Users_TB_July2011.pdfError: User Rate Limit Exceeded kmkaplan
Error: User Rate Limit Exceeded R..
Error: User Rate Limit Exceeded David Schwartz
Error: User Rate Limit ExceededcertificationError: User Rate Limit Exceeded R..

Ваш Ответ

4   ответа
7

Edit:Error: User Rate Limit Exceeded

Error: User Rate Limit ExceededatomicError: User Rate Limit Exceededconcurrent writes will not tear with respect to one another on any major OSError: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

All of the following functions shall be atomic with respect to each other in the effects specified in POSIX.1-2008 when they operate on regular files or symbolic links ... [many functions] ... read() ... write() ... If two threads each call one of these functions, each call shall either see all of, the specified effects of the other call, or none of them. [Source]

Error: User Rate Limit Exceeded

Writes can be serialized with respect to other reads and writes. If a read() of file data can be proven (by any means) to occur after a write() of the data, it must reflect that write(), even if the calls are made by different processes. [Source]

Error: User Rate Limit Exceeded

This volume of POSIX.1-2008 does not specify behavior of concurrent writes to a file from multiple processes. Applications should use some form of concurrency control. [Source]

Error: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

Error: User Rate Limit ExceededError: User Rate Limit ExceededError: User Rate Limit Exceeded


No O_DIRECT/FILE_FLAG_NO_BUFFERING:

Error: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

O_DIRECT/FILE_FLAG_NO_BUFFERING:

Error: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

Error: User Rate Limit Exceeded


Error: User Rate Limit Exceeded

Error: User Rate Limit ExceededError: User Rate Limit ExceededError: User Rate Limit Exceeded

Причина, по которой сквозная запись делает атомарность обновлений меньшей, вероятно, заключается в том, что Microsoft реализовала быстрый путь DMA и медленный путь не-DMA для обновлений, а когда включена сквозная запись, она использует медленный путь, который вообще не использует DMA, и / о, вероятно, с помощью опроса регистра. Учитывая удивительно медленный fsync и то, как он будет доминировать над всем остальным кодом с точки зрения стоимости, Microsoft не чувствовала необходимости ускорять процесс записи через код. В конце концов, только Microsoft знает наверняка, мой эмпирический тестер просто показывает атомарность, а не причины, почему и почему нет.
Вы хотели сказать 512 байт, если FILE_FLAG_WRITE_THROUGHon? Если нет, то почему для этого флага имеет смысл усугублять ситуацию?
8

Error: User Rate Limit Exceeded

Handles can be created or destroyed by explicit user action, without affecting the underlying open file description. Some of the ways to create them include fcntl(), dup(), fdopen(), fileno(), and fork(). They can be destroyed by at least fclose(), close(), and the exec functions. [ ... ] Note that after a fork(), two handles exist where one existed before.

Error: User Rate Limit ExceededforkError: User Rate Limit Exceededfork()Error: User Rate Limit Exceeded

The child process shall have its own copy of the parent's file descriptors. Each of the child's file descriptors shall refer to the same open file description with the corresponding file descriptor of the parent.

Error: User Rate Limit Exceeded

  • the child has copies of the parent's file descriptors
  • the child's copies refer to the same "thing" that the parent can access via said fds
  • file descriptors and file descriptions are not the same thing; in particular, a file descriptor is a handle in the above sense.

Error: User Rate Limit Exceededfork()Error: User Rate Limit ExceededcopiesError: User Rate Limit ExceededdetachedError: User Rate Limit Exceeded

Error: User Rate Limit ExceededprocessError: User Rate Limit Exceededindependent instancesError: User Rate Limit Exceededwrite()Error: User Rate Limit Exceeded

On a regular file or other file capable of seeking, the actual writing of data shall proceed from the position in the file indicated by the file offset associated with fildes. Before successful return from write(), the file offset shall be incremented by the number of bytes actually written.

Error: User Rate Limit ExceededcopyError: User Rate Limit ExceededNError: User Rate Limit Exceededexactly NError: User Rate Limit Exceeded0 <=Error: User Rate Limit Exceeded<= NError: User Rate Limit ExceededincrementedError: User Rate Limit Exceeded

Error: User Rate Limit Exceeded

Error: User Rate Limit ExceededO_APPENDError: User Rate Limit Exceededwrite()Error: User Rate Limit Exceeded

If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation.

Error: User Rate Limit Exceeded

Error: User Rate Limit ExceededthreadsError: User Rate Limit ExceededcopiesError: User Rate Limit Exceeded

MoralError: User Rate Limit Exceededrestrictive by defaultError: User Rate Limit Exceededyou as the programmerError: User Rate Limit Exceeded

10

man 2 writeError: User Rate Limit Exceeded

Note that not all file systems are POSIX conforming.

Error: User Rate Limit ExceededError: User Rate Limit ExceededError: User Rate Limit Exceededext4Error: User Rate Limit Exceeded

Currently concurrent reads/writes are atomic only wrt individual pages, however are not on the system call. This may cause read() to return data mixed from several different writes, which I do not think it is good approach. We might argue that application doing this is broken, but actually this is something we can easily do on filesystem level without significant performance issues, so we can be consistent. Also POSIX mentions this as well and XFS filesystem already has this feature.

Error: User Rate Limit Exceededext4Error: User Rate Limit Exceeded

Хотя это сильно огорчает меня, чем больше я копаюсь в этом, тем больше кажется, что ты прав. kmkaplan
6

Error: User Rate Limit Exceeded

Either a file descriptor or a stream is called a "handle" on the open file description to which it refers; an open file description may have several handles. […] All activity by the application affecting the file offset on the first handle shall be suspended until it again becomes the active file handle. […] The handles need not be in the same process for these rules to apply.

Error: User Rate Limit Exceeded

Error: User Rate Limit ExceededPIPE_BUF.

Error: User Rate Limit ExceededwriteError: User Rate Limit ExceededPIPE_BUF, writeError: User Rate Limit ExceededwriteError: User Rate Limit Exceeded

Случай для коротких записей обрабатывается в предоставленном примере. Сейчас я перечитываю документ с учетом вашей интерпретации & # x2026; Я посмотрю, что это даст. kmkaplan
& # x201C; Все действия приложения, влияющие на смещение файла в первом дескрипторе, должны быть приостановлены до тех пор, пока он снова не станет активным дескриптором файла & # x201D ;. Как можно указать требование в заявке? Если он приостанавливает все действия, влияющие на смещение файла, то в случае, когда один из дескрипторов является потоком, он никогда не сможет выполнить функцию [lf] seek (), необходимую для того, чтобы & # x201C; снова стал дескриптором активного файла & # x201D ;. kmkaplan
Все, что говорит о том, что вы не можете вызывать функции поиска для неактивного дескриптора. Этоactive handle что вам может потребоваться вызвать функцию поиска, чтобы переключиться на новый дескриптор, и это совершенно законно.
Полагаю, мне следовало сказать «когда активен другой дескриптор».
& # x201C; вам не разрешено вызывать функции поиска для неактивного дескриптора & # x201D; противоречит & # x201C; чтобы дескриптор стал активным дескриптором [& # x2026;], приложение должно выполнить lseek () или fseek () [для него] & # x201D; kmkaplan

Похожие вопросы