Project

General

Profile

Actions

Bug #3981

closed

Groups with large amount of users causing massive performance issues

Added by krileon over 9 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Immediate
Assignee:
Target version:
Start date:
19 April 2013
Due date:
% Done:

100%

Estimated time:

Description

Create a group and 12,000 fake users then have those users join the group. This will kill a sites performance. Even if memory limit is pushed to 2GB. This needs to be deep profiled to find where the memory leak is.

https://www.joomlapolis.com/forum/153-professional-member-support/220812-fatal-error-allowed-memory-size-of-536870912-bytes-exhausted-tried-to-allocate-32-bytes

Actions #1

Updated by krileon over 9 years ago

Appears to cost 8MB of memory per 200 users. This appears to be far too much. At most should be 1MB or 0.5MB. At this rate when reaching the thousands mark you'll be exhausting far too much memory.

Actions #2

Updated by krileon over 9 years ago

Appears to be due to query calls. A group with 1,000 users has over 2,000 queries. Many repeating of the following.

SELECT c.*, u.*
FROM jos_users AS u
LEFT JOIN jos_comprofiler AS c
ON c.id = u.id
WHERE u.id = USERID
SELECT a.id
FROM jos_user_usergroup_map AS map
LEFT JOIN jos_usergroups AS a
ON a.id = map.group_id
WHERE map.user_id = USERID

They appear to be ACL related.

Actions #3

Updated by krileon over 9 years ago

This is due to calling get_groups_below_me for each users in the Users tab inside of getAuthorization when adding authorization checks to the data call. Once the authorization checks are removed the issue is completely gone.

The first query comes from CB it self and the second comes from Joomla. Need to be able to get groups below a user without causing so much overhead even though the results are cached.

Actions #4

Updated by krileon over 9 years ago

Second issue is get_user_moderator as it's also running get_groups_below_me. So a more performance optimized method for checking if a user is a moderator is needed as well.

Actions #5

Updated by krileon over 9 years ago

It's due to the following.

if ( ( ! $owner ) && method_exists( $row, 'getOwner' ) ) {
$owner = $row->getOwner();
}

This is called in the access function of cbgjData. Once removed performance is fine. It's used in the authorization function to see if the owner is the viewing user. So maybe this should be improved to only need the user id as that's all that is being checked?

Actions #6

Updated by krileon over 9 years ago

The above doesn't get rid of all the memory usage (you will have memory usage increase with more users, it's just logical that it does), but it does remove a good chunk of the memory usage as well as 2,000 queries is reduced to 100.

Actions #7

Updated by krileon over 9 years ago

  • Status changed from Assigned to Resolved
  • % Done changed from 0 to 100

This appears to have resolved the issue. It does not seam to be possible to improve memory usage any further with the current usage.

Actions #8

Updated by krileon about 9 years ago

  • Target version set to 2.8.0
Actions #9

Updated by krileon about 7 years ago

  • Target version changed from 2.8.0 to 3.0.0
Actions #10

Updated by krileon over 6 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF