Friday, July 4, 2014

Operating System Interview Question: Inter-Process Communication

inter-process communication (IPC) is a set of methods for the exchange of data among multiple threads in one or more processes. Processes may be running on one or more computers connected by a network. IPC methods are divided into methods for message passingsynchronizationshared memory, and remote procedure calls (RPC).

Anonymous Pipeline 
A two-way data stream interfaced through standard input and output and is read character by character.
a set of processes chained by their standard streams, so that the output of each process (stdout) feeds directly as input (stdin) to the next one. Each connection is implemented by an anonymous pipe.

ls -l | grep key | less

Named pipe
A pipe implemented through a file on the file system instead of standard input and output.

Named pipe makes use of the filesystem. It is explicitly created using mkfifo(), and two separate processes can access the pipe by name — one process can open it as a reader, and the other as a writer.

A traditional pipe is "unnamed" because it exists anonymously and persists only for as long as the process is running. 

A named pipe is system-persistent and exists beyond the life of the process and can be deleted once it is no longer being used. Processes generally attach to the named pipes (usually appearing as a file) to perform inter-process communication.

mkfifo my_pipe
gzip -9 -c < my_pipe > out.gz &

cat file > my_pipe
rm my_pipe

A named pipe can be used to transfer information from one application to another without the use of an intermediate temporary file.

In windows, anonymous pipes used in pipelining are actually named pipes with a random name.

We can use socket to send data over network interface to a different process on the same computer or to another computer.

Message queues
Message queues provide an asynchronous communications protocol, meaning that the sender and receiver of the message do not need to interact with the message queue at the same time. Messages placed onto the queue are stored until the recipient retrieves them. 
Examples of informations include Apache ActiveMQ, IBM WebSphere MQ.

Shared memory
Shared memory is memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them or avoid redundant copies

a semaphore is a variable or abstract data type that is used for controlling access, by multiple processes, to a common resource in a parallel programming environment.
Semaphores which allow an arbitrary resource count are called counting semaphores, while semaphores which are restricted to the values 0 and 1 (or locked/unlocked, unavailable/available) are called binary semaphores.

Counting semaphores are equipped with two operations, historically denoted as V (also known as signal) and P (or wait). Operation V increments the semaphore S, and operation P decrements it.

A signal is an asynchronous notification sent to a process or to a specific thread within the same process in order to notify it of an event that occurred.
When a signal is sent, the operating system interrupts the target process's normal flow of execution to deliver the signal. Execution can be interrupted during any non-atomic instruction. If the process has previously registered a signal handler, that routine is executed. Otherwise, the default signal handler is executed.

For example, the kill command send a specified signal to a specified process. 
Ctrl-C (in older Unixes, DEL) sends an INT signal (SIGINT); by default, this causes the process to terminate.

We can use file to communicate between multiple process.

Memory-mapped file
A file mapped to RAM and can be modified by changing memory addresses directly instead of outputting to a stream, shares same benefits as a standard file.

Semaphores vs. mutexes
Mutexes have a concept of an owner, which is the process that locked the mutex. Only the process that locked the mutex can unlock it. 
In contrast, a semaphore has no concept of an owner. Any process can unlock a semaphore.

Unlike semaphores, mutexes provide priority inversion safety. Since the mutex knows its current owner, it is possible to promote the priority of the owner whenever a higher-priority task starts waiting on the mutex.

Mutexes also provide deletion safety, where the process holding the mutex cannot be accidentally deleted. Semaphores do not provide this.
MethodShort DescriptionProvided by (operating systems or other environments)
FileA record stored on disk, or a record synthesized on demand by a file server, which can be accessed by multiple processes.Most operating systems
SignalA system message sent from one process to another, not usually used to transfer data but instead used to remotely command the partnered process.Most operating systems
SocketA data stream sent over a network interface, either to a different process on the same computer or to another computer on the network.Most operating systems
Message queueAn anonymous data stream similar to a socket, usually implemented by the operating system, that allows multiple processes to read and write to the message queue without being directly connected to each other.Most operating systems
PipeA two-way data stream between two processes interfaced through standard input and output and read in one character at a time.All POSIX systems, Windows
Named pipeA pipe implemented through a file on the file system instead of standard input and output. Multiple processes can read and write to the file as a buffer for IPC data.All POSIX systems, Windows
SemaphoreA simple structure that synchronizes multiple processes acting on shared resources.All POSIX systems, Windows
Shared memoryMultiple processes are given access to the same block of memory which creates a shared buffer for the processes to communicate with each other.All POSIX systems, Windows
Message passingAllows multiple programs to communicate using channels, commonly used in concurrency models.Used in MPI paradigm, Java RMICORBADDSMSMQ,MailSlotsQNX, others
Memory-mapped fileA file mapped to RAM and can be modified by changing memory addresses directly instead of outputting to a stream. This shares the same benefits as a standard file.All POSIX systems, Windows
Read full article from Inter-process communication - Wikipedia, the free encyclopedia

No comments:

Post a Comment


Review (554) System Design (293) System Design - Review (189) Java (178) Coding (75) Interview-System Design (65) Interview (60) Book Notes (59) Coding - Review (59) to-do (45) Knowledge (39) Linux (39) Interview-Java (35) Knowledge - Review (32) Database (30) Design Patterns (29) Product Architecture (28) Big Data (27) Soft Skills (27) Miscs (25) MultiThread (25) Concurrency (24) Cracking Code Interview (24) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Distributed (20) Interview Q&A (20) OOD Design (20) System Design - Practice (19) Security (17) Algorithm (15) How to Ace Interview (15) Brain Teaser (14) Google (13) Linux - Shell (13) Spark (13) Spring (13) Code Quality (12) How to (12) Interview-Database (12) Interview-Operating System (12) Redis (12) Tools (12) Architecture Principles (11) Company - LinkedIn (11) Testing (11) Resource (10) Solr (10) Amazon (9) Cache (9) Search (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Company - Uber (8) Interview - MultiThread (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Scalability (8) Cassandra (7) Git (7) Interview Corner (7) JVM (7) Java Basics (7) Machine Learning (7) NoSQL (7) C++ (6) Design (6) File System (6) Highscalability (6) How to Better (6) Kafka (6) Network (6) Restful (6) Trouble Shooting (6) CareerCup (5) Code Review (5) Company - Facebook (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Be Architect (4) Big Fata (4) C (4) Company Product Architecture (4) Data structures (4) Design Principles (4) Facebook (4) GeeksforGeeks (4) Generics (4) Google Interview (4) Hardware (4) JDK8 (4) Optimization (4) Product + Framework (4) Shopping System (4) Source Code (4) Web Service (4) node.js (4) Back-of-Envelope (3) Company - Pinterest (3) Company - Twiiter (3) Company - Twitter (3) Consistent Hash (3) GOF (3) Game Design (3) GeoHash (3) Growth (3) Guava (3) Interview-Big Data (3) Interview-Linux (3) Interview-Network (3) Java EE Patterns (3) Javarevisited (3) Map Reduce (3) Math - Probabilities (3) Performance (3) Puzzles (3) Python (3) Resource-System Desgin (3) Scala (3) UML (3) geeksquiz (3) AI (2) API Design (2) AngularJS (2) Behavior Question (2) Bugs (2) Coding Interview (2) Company - Netflix (2) Crawler (2) Cross Data Center (2) Data Structure Design (2) Database-Shard (2) Debugging (2) Docker (2) Elasticsearch (2) Garbage Collection (2) Go (2) Hadoop (2) Html (2) Interview - Soft Skills (2) Interview-Miscs (2) Interview-Web (2) JDK (2) Logging (2) POI (2) Papers (2) Programming (2) Project Practice (2) Random (2) Software Desgin (2) System Design - Feed (2) Thread Synchronization (2) Video (2) ZooKeeper (2) reddit (2) Ads (1) Advanced data structures (1) Algorithm - Review (1) Android (1) Approximate Algorithms (1) Base X (1) Bash (1) Books (1) C# (1) CSS (1) Chrome (1) Client-Side (1) Cloud (1) CodingHorror (1) Company - Yelp (1) Counter (1) DSL (1) Dead Lock (1) Difficult Puzzles (1) Distributed ALgorithm (1) Eclipse (1) Facebook Interview (1) Function Design (1) Functional (1) GoLang (1) How to Solve Problems (1) ID Generation (1) IO (1) Important (1) Internals (1) Interview - Dropbox (1) Interview - Project Experience (1) Interview Tips (1) Interview-Brain Teaser (1) Interview-How (1) Interview-Mics (1) Interview-Process (1) Jeff Dean (1) Joda (1) LeetCode - Review (1) Library (1) LinkedIn (1) LintCode (1) Mac (1) Micro-Services (1) Mini System (1) MySQL (1) Nigix (1) NonBlock (1) Process (1) Productivity (1) Program Output (1) Programcreek (1) Quora (1) RPC (1) Raft (1) RateLimiter (1) Reactive (1) Reading (1) Reading Code (1) Refactoring (1) Resource-Java (1) Resource-System Design (1) Resume (1) SQL (1) Sampling (1) Shuffle (1) Slide Window (1) Spotify (1) Stability (1) Storm (1) Summary (1) System Design - TODO (1) Tic Tac Toe (1) Time Management (1) Web Tools (1) algolist (1) corejavainterviewquestions (1) martin fowler (1) mitbbs (1)

Popular Posts