Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Permission search takes too long #53

Open
chris-rock opened this issue Mar 16, 2017 · 14 comments
Open

Permission search takes too long #53

chris-rock opened this issue Mar 16, 2017 · 14 comments

Comments

@chris-rock
Copy link
Member

chris-rock commented Mar 16, 2017 •
edited

Customers reported that the find permission command takes too long. We should find a solution to do this faster.

Running handlers:
[2017-03-16T13:17:19-04:00] INFO: Running report handlers
[2017-03-16T13:17:19-04:00] WARN: Format is json
[2017-03-16T13:17:19-04:00] INFO: Initialize InSpec
[2017-03-16T13:17:20-04:00] WARN: URL target https://github.com/dev-sec/linux-baseline transformed to https://github.com/dev-sec/linux-baseline/archive/master.tar.gz. Consider using the git fetcher
[2017-03-16T13:17:20-04:00] INFO: Running tests from: [{:name=>"linux-baseline", :supermarket=>"dev-sec/linux-baseline"}]
 
 
 
 
 
 
[2017-03-16T13:27:30-04:00] ERROR: Report handler Chef::Handler::AuditReport raised #<Mixlib::ShellOut::CommandTimeout: Command timed out after 600s:
Command exceeded allowed execution time, process terminated
---- Begin output of find / -perm -4000 -o -perm -2000 -type f ! -path '/proc/*' ! -path '/var/lib/lxd/containers/*' -print 2>/dev/null | grep -v '^find:' ----
STDOUT:
STDERR:
---- End output of find / -perm -4000 -o -perm -2000 -type f ! -path '/proc/*' ! -path '/var/lib/lxd/containers/*' -print 2>/dev/null | grep -v '^find:' ----
Ran find / -perm -4000 -o -perm -2000 -type f ! -path '/proc/*' ! -path '/var/lib/lxd/containers/*' -print 2>/dev/null | grep -v '^find:' returned >
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:124:in `run_command'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:259:in `run_command'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/train-0.22.1/lib/train/transports/local.rb:32:in `run_command'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/resources/command.rb:31:in `result'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/resources/command.rb:35:in `stdout'
[2017-03-16T13:27:30-04:00] ERROR: linux-baseline-master/controls/os_spec.rb:193:in `block in load_with_context'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/rule.rb:51:in `instance_eval'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/rule.rb:51:in `initialize'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/control_eval_context.rb:71:in `new'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/control_eval_context.rb:71:in `block (2 levels) in create'
[2017-03-16T13:27:30-04:00] ERROR: linux-baseline-master/controls/os_spec.rb:187:in `load_with_context'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/profile_context.rb:146:in `instance_eval'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/profile_context.rb:146:in `load_with_context'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/profile_context.rb:130:in `load_control_file'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/profile.rb:144:in `block in collect_tests'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/profile.rb:141:in `each'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/profile.rb:141:in `collect_tests'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/runner.rb:90:in `block in load'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/runner.rb:79:in `each'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/runner.rb:79:in `load'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.15.0/lib/inspec/runner.rb:100:in `run'
[2017-03-16T13:27:30-04:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:116:in `call'
[2017-03-16T13:27:30-04:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:47:in `block in report'
[2017-03-16T13:27:30-04:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:33:in `each'
[2017-03-16T13:27:30-04:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:33:in `report'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:259:in `run_report_unsafe'
[2017-03-16T13:27:30-04:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:59:in `run_report_safely'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:125:in `block in run_report_handlers'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:123:in `each'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:123:in `run_report_handlers'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:135:in `block in <class:Handler>'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:441:in `block in run_completed_successfully'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:440:in `each'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:440:in `run_completed_successfully'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:299:in `run'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:295:in `block in fork_chef_client'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork_chef_client'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:248:in `block in run_chef_client'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/local_mode.rb:44:in `with_server_connectivity'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:236:in `run_chef_client'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `loop'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `interval_run_chef_client'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:434:in `run_application'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:59:in `run'
[2017-03-16T13:27:30-04:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/bin/chef-client:26:in `<top (required)>'
[2017-03-16T13:27:30-04:00] ERROR: /bin/chef-client:57:in `load'
[2017-03-16T13:27:30-04:00] ERROR: /bin/chef-client:57:in `<main>'
  - Chef::Handler::AuditReport
Running handlers complete
[2017-03-16T13:27:30-04:00] INFO: Report handlers complete
Chef Client finished, 1/11 resources updated in 10 minutes 26 seconds
@artem-sidorenko
Copy link
Member

If it possible to get a bit more information here? My assumption is that this find goes through a filesystem with a huge amount of files (some filesystem with tons of data)

I faced once a similar problem with my first implementations of puppet-os-hardening at P&I years ago and we resolved it this way: exclude filesystems with nosuid, nodev, noexec mount options from a such find and mount all data filesystems with this mount options.

@mcgege
Copy link
Member

mcgege commented Jul 13, 2017 •
edited

For the search in os-01 and os-09 I would recommend to limit the the search depth with the option -maxdepth. IMHO 3 levels should be sufficient ... for os-06 I don't see a way to optimize the search.

BTW: Why do you search for .rhosts in os-01 and also in os-09?

@atomic111
Copy link
Member

@mcgege thanks for your feedback. i agree with you. it should be something like this:

describe file('/etc/hosts.equiv') do
  it { should_not exist }
end

and to limit the max depth to 3 should be sufficient. Can you create a PR for this?

@mcgege
Copy link
Member

mcgege commented Jul 13, 2017

@atomic111 Of course! See #77

@mike-stewart
Copy link

mike-stewart commented Jul 13, 2017 •
edited

I'm getting this error when running inspec compliance upload linux-baseline/. Why is that being run during the upload? Any workaround?

I, [2017-07-13T15:14:25.786924 #90832]  INFO -- : Checking profile in ../linux-baseline/
I, [2017-07-13T15:14:25.811868 #90832]  INFO -- : Metadata OK.
/opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:124:in `run_command': Command timed out after 600s: (Mixlib::ShellOut::CommandTimeout)
Command exceeded allowed execution time, process terminated
---- Begin output of find / -perm -4000 -o -perm -2000 -type f ! -path '/proc/*' ! -path '/var/lib/lxd/containers/*' -print 2>/dev/null | grep -v '^find:' ----
STDOUT: 
STDERR: 
---- End output of find / -perm -4000 -o -perm -2000 -type f ! -path '/proc/*' ! -path '/var/lib/lxd/containers/*' -print 2>/dev/null | grep -v '^find:' ----
Ran find / -perm -4000 -o -perm -2000 -type f ! -path '/proc/*' ! -path '/var/lib/lxd/containers/*' -print 2>/dev/null | grep -v '^find:' returned 
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:259:in `run_command'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/train-0.25.0/lib/train/transports/local.rb:32:in `run_command'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/resources/command.rb:33:in `result'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/resources/command.rb:37:in `stdout'
	from ../linux-baseline/controls/os_spec.rb:189:in `block in load_with_context'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/rule.rb:49:in `instance_eval'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/rule.rb:49:in `initialize'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/control_eval_context.rb:71:in `new'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/control_eval_context.rb:71:in `block (2 levels) in create'
	from ../linux-baseline/controls/os_spec.rb:183:in `load_with_context'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile_context.rb:146:in `instance_eval'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile_context.rb:146:in `load_with_context'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile_context.rb:130:in `load_control_file'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:151:in `block in collect_tests'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:148:in `each'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:148:in `collect_tests'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:454:in `load_checks_params'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:447:in `load_params'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:141:in `params'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:307:in `controls_count'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/inspec/profile.rb:278:in `check'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/lib/bundles/inspec-compliance/cli.rb:186:in `upload'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/invocation.rb:115:in `invoke'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor.rb:235:in `block in subcommand'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
	from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.30.0/bin/inspec:12:in `<top (required)>'
	from /usr/local/bin/inspec:264:in `load'
	from /usr/local/bin/inspec:264:in `<main>'

@atomic111
Copy link
Member

@mike-stewart i tested it and it is working. i used the inspec version 1.31.1 and the compliance server 1.10.2

i included you my commands, which i used to upload the linux-baseline.

atomic111:..ooks/linux-baseline ±> inspec compliance login https://192.168.100.40 --insecure --user=creator --refresh-token=<refresh-token>
WARN: Unresolved specs during Gem::Specification.reset:
      thor (~> 0.19)
      rspec (~> 3)
      addressable (~> 2.4)
      winrm (~> 2.0)
      docker-api (~> 1.26)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.

API access token verified
atomic111:..ooks/linux-baseline ±> inspec compliance upload ./                                                                                                                                      11d [3303c00]
WARN: Unresolved specs during Gem::Specification.reset:
      thor (~> 0.19)
      rspec (~> 3)
      addressable (~> 2.4)
      winrm (~> 2.0)
      docker-api (~> 1.26)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Profile is already vendored. Use --overwrite.
I, [2017-07-14T09:09:10.939861 #24338]  INFO -- : Checking profile in ./
I, [2017-07-14T09:09:10.941846 #24338]  INFO -- : Metadata OK.
I, [2017-07-14T09:09:14.976291 #24338]  INFO -- : Found 52 controls.
I, [2017-07-14T09:09:14.976417 #24338]  INFO -- : Control definitions OK.
Profile is valid
Generate temporary profile archive at /tmp/linux-baseline20170714-24338-u9fe.tar.gz
I, [2017-07-14T09:09:15.126673 #24338]  INFO -- : Generate archive /tmp/linux-baseline20170714-24338-u9fe.tar.gz.
I, [2017-07-14T09:09:15.135052 #24338]  INFO -- : Finished archive generation.
Start upload to creator/linux-baseline
Uploading to Chef Compliance
Successfully uploaded profile
atomic111:..ooks/linux-baseline ±> inspec version                                                                                                                                                   11d [3303c00]
WARN: Unresolved specs during Gem::Specification.reset:
      thor (~> 0.19)
      rspec (~> 3)
      addressable (~> 2.4)
      winrm (~> 2.0)
      docker-api (~> 1.26)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
1.31.1

@mike-stewart
Copy link

@atomic111 Still doesn't seem to be working for me. Is it possible that it's running the find command on my local machine as part of the upload, and it's failing for me because I have a lot of files on my machine?

@artem-sidorenko
Copy link
Member

@mike-stewart it looks like your upload should take some time, can you check in another console with ps axw | grep find or similar if this find is executed really on your system?

@mcgege
Copy link
Member

mcgege commented Aug 12, 2017 •
edited

How about this: use locate instead of find if installed --> if you have timeout problems install the (m)locate package on your system
This might also solve #78

@mattlqx
Copy link
Contributor

mattlqx commented May 31, 2018 •
edited

My solution here was to create a wrapper profile with a find that looks for network fs types and excludes those paths from the find. My hosts from 10 minutes to execute the stock profile down to 30 seconds to execute the wrapper profile.

https://gist.github.com/mattlqx/24c6730d7586e78a23a31353066cb31c

This is the best/simplest way I found to override a part of a resource from another profile, feedback welcome if there are better ways. The dynamic classes that InSpec resources are made trying this... interesting.

@mattlqx mattlqx mentioned this issue May 31, 2018
Closed
@mattlqx
Copy link
Contributor

mattlqx commented May 31, 2018

I riffed a bit on my gist from the prior comment and submitted a pull to just get into here. With it, by default, network filesystems (arbitrarily defined) are now ignored in the find of suid_check. Attributes are provided to override the exclude behavior.

@bbigras
Copy link

bbigras commented Sep 3, 2019

Any progress on this? Or anyway to increase the 600s timeout?

@chris-rock
Copy link
Member Author

Any PR is welcome to improve the situation.

@mattlqx
Copy link
Contributor

mattlqx commented Sep 3, 2019

I've had a pull open for over a year. 🤷🏻‍♂️

I can rebase it. I'm not entirely sure what comments were meant to be actioned on though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add attributes to exclude fstypes and paths from suid_check find
7 participants
@bbigras @mattlqx @chris-rock @artem-sidorenko @mike-stewart @mcgege @atomic111