Skip to content

Commit 3b09dc1

Browse files
lyrixxlyrixx
committed
[Semaphore] Added the component
Few years ago, we have introduced the Lock component. This is a very nice component, but sometime it is not enough. Sometime you need semaphore. This is why I'm introducing this new component. ## What is a Semaphore ? From wikipedia: > In computer science, a semaphore is a variable or abstract data type used to control access to a common resource by multiple processes in a concurrent system such as a multitasking operating system. A semaphore is simply a variable. This variable is used to solve critical section problems and to achieve process synchronization in the multi processing environment. A trivial semaphore is a plain variable that is changed (for example, incremented or decremented, or toggled) depending on programmer-defined conditions. This new component is more than a variable. This is an abstraction on top of different storage. To make a quick comparison with a lock: * A lock allows only 1 process to access a resource; * A semaphore allow N process to access a resource. Basically, a lock is a semaphore where `N = 1`. ### Possible confusion PHP exposes some `sem_*` functions like [`sem_acquire`](http://php.net/sem_acquire). This module provides wrappers for the System V IPC family of functions. It includes semaphores, shared memory and inter-process messaging (IPC). The Lock component has a storage that works with theses functions. It uses it with `N = 1`. ## What are the use-cases ? Wikipedia has some [examples](https://en.wikipedia.org/wiki/Semaphore_(programming)#Examples) But I can add one more commun use case. If you are building an async system that process user data, you may want to priorise all jobs. You can achieve that by running at maximum N jobs per user at the same time. If the user has more resources, you give him more concurrent jobs (so a bigger `N`). Thanks to semaphores, it's pretty easy to know if a new job can be run. ### Some concrete use-cases I'm not saying the following services are using semaphore, but they may solve the previous problematic with semaphores. Here is some examples: * services like testing platform where a user can test N projects concurrently (travis, circle, appveyor, insight, ...) * services that ingest lots of data (newrelic, datadog, blackfire, segment.io, ...)) * services that send email in batch (campaign monitor, mailchimp, ...) * etc... ## How to use it ? To do so, since PHP is mono-threaded, you run M PHP workers. And in each worker, you look for for the next job. When you grab a job, you try to acquires a semaphore. If you got it, you process the job. If not you try another job. FTR in other language, like Go, there are no need to run M workers, one is enough. ### With Symfony ```php <?php use Symfony\Component\Lock\LockFactory; use Symfony\Component\Lock\Store\RedisStore as LockRedisStore; use Symfony\Component\Semaphore\SemaphoreFactory; use Symfony\Component\Semaphore\Store\RedisStore; require __DIR__.'/vendor/autoload.php'; $redis = new Redis(); $redis->connect('172.17.0.2'); // Internally, Semaphore needs a lock $lock = (new LockFactory(new LockRedisStore($redis)))->createLock('test:lock', 1); // Create a semaphore: // * name = test // * limit = 3 (it means only 3 process are allowed) // * ttl = 10 seconds : Maximum expected semaphore duration in seconds $semaphore = (new SemaphoreFactory($lock, new RedisStore($redis)))->createSemaphore('test', 3, 10); if (!$semaphore->acquire()) { echo "Could not acquire the semaphore\n"; exit(1); } // The semaphore has been acquired // Do the heavy job for ($i = 0; $i < 100; ++$i) { sleep(1); // Before the expiration, refresh the semaphore if the job is not finished yet if ($i % 9 === 0) { $semaphore->refresh(); } } // Release it when finished $semaphore->release(); ``` ## Prior art I looked at [packagist](https://packagist.org/?query=semaphore) and: * most of packages are using a semaphore storage for creating a lock. So there are not relevant here; * some packages need an async framework to be used (amphp for example); * the only packages really implementing a semaphore, has a really low code quality and some bugs. ## Current implementation 1. I initially copied the Lock component since the external API is quite similar; 1. I simplified it a lot for the current use case; 1. I implemented the RedisStorage according the [redis book](https://redislabs.com/ebook/part-2-core-concepts/chapter-6-application-components-in-redis/6-3-counting-semaphores/;) 1. I forced a TTL on the storage.
1 parent cde44fc commit 3b09dc1

19 files changed

+871
-0
lines changed

‎src/Symfony/Component/Semaphore/.gitattributes

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
/.gitignore export-ignore
2+
/phpunit.xml.dist export-ignore
3+
/Tests export-ignore

‎src/Symfony/Component/Semaphore/.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
composer.lock
2+
phpunit.xml
3+
vendor/

‎src/Symfony/Component/Semaphore/CHANGELOG.md

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
CHANGELOG
2+
=========
3+
4+
5.1.0
5+
-----
6+
7+
* added the component

‎src/Symfony/Component/Semaphore/Exception/ExceptionInterface.php

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Semaphore\Exception;
13+
14+
/**
15+
* Base ExceptionInterface for the Semaphore Component.
16+
*
17+
* @author Jérémy Derussé <jeremy@derusse.com>
18+
*/
19+
interface ExceptionInterface extends \Throwable
20+
{
21+
}

‎src/Symfony/Component/Semaphore/Exception/InvalidArgumentException.php

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Semaphore\Exception;
13+
14+
/**
15+
* @author Jérémy Derussé <jeremy@derusse.com>
16+
*/
17+
class InvalidArgumentException extends \InvalidArgumentException implements ExceptionInterface
18+
{
19+
}

‎src/Symfony/Component/Semaphore/Exception/SemaphoreAcquiringException.php

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Semaphore\Exception;
13+
14+
/**
15+
* SemaphoreAcquiringException is thrown when an issue happens during the acquisition of a semaphore.
16+
*
17+
* @author Jérémy Derussé <jeremy@derusse.com>
18+
*/
19+
class SemaphoreAcquiringException extends \RuntimeException implements ExceptionInterface
20+
{
21+
}

‎src/Symfony/Component/Semaphore/Exception/SemaphoreExpiredException.php

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Semaphore\Exception;
13+
14+
/**
15+
* SemaphoreExpiredException is thrown when a semaphore may conflict due to a TTL expiration.
16+
*
17+
* @author Jérémy Derussé <jeremy@derusse.com>
18+
*/
19+
class SemaphoreExpiredException extends \RuntimeException implements ExceptionInterface
20+
{
21+
}

‎src/Symfony/Component/Semaphore/Exception/SemaphoreReleasingException.php

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Semaphore\Exception;
13+
14+
/**
15+
* SemaphoreReleasingException is thrown when an issue happens during the release of a semaphore.
16+
*
17+
* @author Jérémy Derussé <jeremy@derusse.com>
18+
*/
19+
class SemaphoreReleasingException extends \RuntimeException implements ExceptionInterface
20+
{
21+
}

‎src/Symfony/Component/Semaphore/Key.php

+94
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Semaphore;
13+
14+
use Symfony\Component\Semaphore\Exception\InvalidArgumentException;
15+
16+
/**
17+
* Key is a container for the state of the semaphores in stores.
18+
*
19+
* @author Grégoire Pineau <lyrixx@lyrixx.info>
20+
* @author Jérémy Derussé <jeremy@derusse.com>
21+
*/
22+
final class Key
23+
{
24+
private $resource;
25+
private $limit;
26+
private $expiringTime;
27+
private $state = [];
28+
29+
public function __construct(string $resource, int $limit)
30+
{
31+
if (1 > $limit) {
32+
throw new InvalidArgumentException("The limit ($limit) should be greater than 0.");
33+
}
34+
$this->resource = $resource;
35+
$this->limit = $limit;
36+
}
37+
38+
public function __toString(): string
39+
{
40+
return $this->resource.':'.$this->limit;
41+
}
42+
43+
public function getLimit(): int
44+
{
45+
return $this->limit;
46+
}
47+
48+
public function hasState(string $stateKey): bool
49+
{
50+
return isset($this->state[$stateKey]);
51+
}
52+
53+
public function setState(string $stateKey, $state): void
54+
{
55+
$this->state[$stateKey] = $state;
56+
}
57+
58+
public function removeState(string $stateKey): void
59+
{
60+
unset($this->state[$stateKey]);
61+
}
62+
63+
public function getState(string $stateKey)
64+
{
65+
return $this->state[$stateKey];
66+
}
67+
68+
/**
69+
* @param float $ttl the expiration delay of semaphores in seconds
70+
*/
71+
public function reduceLifetime(float $ttl)
72+
{
73+
$newTime = microtime(true) + $ttl;
74+
75+
if (null === $this->expiringTime || $this->expiringTime > $newTime) {
76+
$this->expiringTime = $newTime;
77+
}
78+
}
79+
80+
/**
81+
* Returns the remaining lifetime.
82+
*
83+
* @return float|null Remaining lifetime in seconds. Null when the key won't expire.
84+
*/
85+
public function getRemainingLifetime(): ?float
86+
{
87+
return null === $this->expiringTime ? null : $this->expiringTime - microtime(true);
88+
}
89+
90+
public function isExpired(): bool
91+
{
92+
return null !== $this->expiringTime && $this->expiringTime <= microtime(true);
93+
}
94+
}

‎src/Symfony/Component/Semaphore/LICENSE

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Copyright (c) 2016-2020 Fabien Potencier
2+
3+
Permission is hereby granted, free of charge, to any person obtaining a copy
4+
of this software and associated documentation files (the "Software"), to deal
5+
in the Software without restriction, including without limitation the rights
6+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7+
copies of the Software, and to permit persons to whom the Software is furnished
8+
to do so, subject to the following conditions:
9+
10+
The above copyright notice and this permission notice shall be included in all
11+
copies or substantial portions of the Software.
12+
13+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19+
THE SOFTWARE.

‎src/Symfony/Component/Semaphore/PersistingStoreInterface.php

+49
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Semaphore;
13+
14+
use Symfony\Component\Semaphore\Exception\SemaphoreAcquiringException;
15+
use Symfony\Component\Semaphore\Exception\SemaphoreExpiredException;
16+
use Symfony\Component\Semaphore\Exception\SemaphoreReleasingException;
17+
18+
/**
19+
* @author Grégoire Pineau <lyrixx@lyrixx.info>
20+
* @author Jérémy Derussé <jeremy@derusse.com>
21+
*/
22+
interface PersistingStoreInterface
23+
{
24+
/**
25+
* Stores the resource if the semaphore is not full.
26+
*
27+
* @throws SemaphoreAcquiringException
28+
*/
29+
public function save(Key $key, float $ttlInSecond);
30+
31+
/**
32+
* Removes a resource from the storage.
33+
*
34+
* @throws SemaphoreReleasingException
35+
*/
36+
public function delete(Key $key);
37+
38+
/**
39+
* Returns whether or not the resource exists in the storage.
40+
*/
41+
public function exists(Key $key): bool;
42+
43+
/**
44+
* Extends the TTL of a resource.
45+
*
46+
* @throws SemaphoreExpiredException
47+
*/
48+
public function putOffExpiration(Key $key, float $ttlInSecond);
49+
}

‎src/Symfony/Component/Semaphore/README.md

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Semaphore Component
2+
===================
3+
4+
Resources
5+
---------
6+
7+
* [Documentation](https://symfony.com/doc/master/components/semaphore.html)
8+
* [Contributing](https://symfony.com/doc/current/contributing/index.html)
9+
* [Report issues](https://github.com/symfony/symfony/issues) and
10+
[send Pull Requests](https://github.com/symfony/symfony/pulls)
11+
in the [main Symfony repository](https://github.com/symfony/symfony)

0 commit comments

Comments
 (0)