10000 fiber: release lock in pool:put(conn) by romanhabibov · Pull Request #55 · tarantool/mysql · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fiber: release lock in pool:put(conn) #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 2, 2021
Merged

fiber: release lock in pool:put(conn) #55

merged 1 commit into from
Mar 2, 2021

Conversation

romanhabibov
Copy link
Contributor
@romanhabibov romanhabibov commented Dec 17, 2020

@ChangeLog

@romanhabibov romanhabibov changed the title Fiber fiber: release lock in pool:put(conn) Dec 18, 2020
@LeonidVas LeonidVas linked an issue Dec 22, 2020 that may be closed by this pull request
Copy link
Contributor
@LeonidVas LeonidVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! Thank you for the patch.
I don't understand, why you choose api as subsystem? You don't change API in the patch.
Please add a detailed description of the situation that could lead to infinite blocking instead of "case in tests".
See some comments below.

Copy link
Contributor
@LeonidVas LeonidVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you move the second test into a separate commit?

@LeonidVas
Copy link
Contributor

Please, add @ChangeLog for the patchset.

@LeonidVas
Copy link
Contributor
LeonidVas commented Dec 22, 2020

Please give your branches a descriptive name according to SOP(for the future).

@romanhabibov romanhabibov force-pushed the fiber branch 3 times, most recently from 5bfab0c to 6382545 Compare December 24, 2020 09:42
@romanhabibov
Copy link
Contributor Author

Please, add @ChangeLog for the patchset.

Done.

@romanhabibov
Copy link
Contributor Author

Please give your branches a descriptive name according to SOP(for the future).

Ok.
I thought this naming was in order not to get confused in the branches, if there are a lot of them. And this repository is not very rich in branches.

@romanhabibov
Copy link
Contributor Author

Why did you move the second test into a separate commit?

Because this test is not about pool connections, but my first patch is about. Also, this test does not fail on master, it was added, because Sanya (@Totktonada) requested it in the issue description.

Copy link
Contributor
@LeonidVas LeonidVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! This iteration looks better than the previous one.
See some comments below.

@LeonidVas
Copy link
Contributor

Please, add @ChangeLog for the patchset.

Done.

Optional!!!
When the user reads the release notes it will be nice if he understand: how the behavior has been changed. Try to write more detailed description.

Copy link
Contributor
@LeonidVas LeonidVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! As for me now the test seems not bad. I think the result could be something like this:

--- gh-34: Check that fiber is not blocked in the following case.
-- A connection conn is acquired from a pool and we acquire a
-- lock. Then we start the first fiber with pool:put(conn)`, which
-- try to acquire lock and yield. Then the second fiber is started
-- to perform request with `conn:execute()` which try to acquire
-- lock, because the connection is still marked as `usable` and
-- yield too. It's appeared that the lock is never released. Now,
-- it's fixed.
local function test_block_fiber_inf(test, pool)
    test:plan(2)
    local conn = pool:get()

    local function fiber_block_request(conn)
        local res, err = pcall(conn.execute, conn, 'SELECT "a"')
        test:ok(res == false and string.find(err, 'Connection is broken'),
            'query failed')
    end

    conn.queue:get()
    fiber.create(pool.put, pool, conn)
    local blocked_fiber = fiber.create(fiber_block_request, conn)
    conn.queue:put(true)

    local start_time = fiber.time()
    local timeout = 5
    local is_alive = true

    -- Wait for the blocked fiber to finish,
    -- but no more than "timeout" seconds.
    while fiber.time() - start_time < timeout and is_alive do
        fiber.sleep(0.1)
        is_alive = blocked_fiber:status() ~= 'dead'
    end

    test:ok(is_alive == false, 'fiber is not blocked')
end

See some comments below.

@romanhabibov
Copy link
Contributor Author

Hi! As for me now the test seems not bad. I think the result could be something like this:

--- gh-34: Check that fiber is not blocked in the following case.
-- A connection conn is acquired from a pool and we acquire a
-- lock. Then we start the first fiber with pool:put(conn)`, which
-- try to acquire lock and yield. Then the second fiber is started
-- to perform request with `conn:execute()` which try to acquire
-- lock, because the connection is still marked as `usable` and
-- yield too. It's appeared that the lock is never released. Now,
-- it's fixed.
local function test_block_fiber_inf(test, pool)
    test:plan(2)
    local conn = pool:get()

    local function fiber_block_request(conn)
        local res, err = pcall(conn.execute, conn, 'SELECT "a"')
        test:ok(res == false and string.find(err, 'Connection is broken'),
            'query failed')
    end

    conn.queue:get()
    fiber.create(pool.put, pool, conn)
    local blocked_fiber = fiber.create(fiber_block_request, conn)
    conn.queue:put(true)

    local start_time = fiber.time()
    local timeout = 5
    local is_alive = true

    -- Wait for the blocked fiber to finish,
    -- but no more than "timeout" seconds.
    while fiber.time() - start_time < timeout and is_alive do
        fiber.sleep(0.1)
        is_alive = blocked_fiber:status() ~= 'dead'
    end

    test:ok(is_alive == false, 'fiber is not blocked')
end

See some comments below.

I'm okay with your test version. I used it.

Copy link
Contributor
@LeonidVas LeonidVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

F438
@Totktonada
Copy link
Member

I have no objections aside of naming and the forgotten print. I looked at the code and didn't look at the test again (I did it the past time).

Please, rebase the patch upward master.

That's all. I think we can push after that.

@romanhabibov
Copy link
Contributor Author

Rebased. Let's push?

@Totktonada
Copy link
Member

Waiting for PR #58 to run tests.

Make lock arithmetic proper and complite in conn:execute(),
conn:close(), conn:reset() and conn:quote(). Release lock in
pool:put(conn). The absence of this operation could lead to an
infinite bloсking when used with fiber as describerd in the case
below.

A connection conn is acquired from a pool and we acquire a lock.
Then we start the first fiber with pool:put(conn)`, which try to
acquire lock and yield. Then the second fiber is started to
perform conn:execute(), conn:reset() and conn:quote() which try
to acquire lock, because the connection is still marked as
`usable` and yield too. Before this patch, it's appeared that the
lock is never released.

Closes #34
@Totktonada
Copy link
Member
Totktonada commented Mar 2, 2021

Rebased on top of the latest master.

@Totktonada Totktonada merged commit 8a837fa into master Mar 2, 2021
@Totktonada Totktonada deleted the fiber branch March 2, 2021 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure <connection object>:execute() don't block a fiber infinitely
3 participants
0