Showing posts with label file. Show all posts
Showing posts with label file. Show all posts

Monday, March 27, 2017

Sorting files according to size recursively

http://unix.stackexchange.com/questions/88065/sorting-files-according-to-size-recursively



51
down voteaccepted
You can also do this with just du. Just to be on the safe side I'm using this version of du:
$ du --version
du (GNU coreutils) 8.5
The approach:
$ du -ah ..DIR.. | grep -v "/$" | sort -rh

Breakdown of approach

The command du -ah DIR will produce a list of all the files and directories in a given directory DIR. The -h will produce human readable sizes which I prefer. If you don't want them then drop that switch. I'm using the head -6 just to limit the amount of output!
$ du -ah ~/Downloads/ | head -6
4.4M    /home/saml/Downloads/kodak_W820_wireless_frame/W820_W1020_WirelessFrames_exUG_GLB_en.pdf
624K    /home/saml/Downloads/kodak_W820_wireless_frame/easyshare_w820.pdf
4.9M    /home/saml/Downloads/kodak_W820_wireless_frame/W820_W1020WirelessFrameExUG_GLB_en.pdf
9.8M    /home/saml/Downloads/kodak_W820_wireless_frame
8.0K    /home/saml/Downloads/bugs.xls
604K    /home/saml/Downloads/netgear_gs724t/GS7xxT_HIG_5Jan10.pdf
Easy enough to sort it smallest to biggest:
$ du -ah ~/Downloads/ | sort -h | head -6
0   /home/saml/Downloads/apps_archive/monitoring/nagios/nagios-check_sip-1.3/usr/lib64/nagios/plugins/check_ldaps
0   /home/saml/Downloads/data/elasticsearch/nodes/0/indices/logstash-2013.04.06/0/index/write.lock
0   /home/saml/Downloads/data/elasticsearch/nodes/0/indices/logstash-2013.04.06/0/translog/translog-1365292480753
0   /home/saml/Downloads/data/elasticsearch/nodes/0/indices/logstash-2013.04.06/1/index/write.lock
0   /home/saml/Downloads/data/elasticsearch/nodes/0/indices/logstash-2013.04.06/1/translog/translog-1365292480946
0   /home/saml/Downloads/data/elasticsearch/nodes/0/indices/logstash-2013.04.06/2/index/write.lock
Reverse it, biggest to smallest:
$ du -ah ~/Downloads/ | sort -rh | head -6
10G /home/saml/Downloads/
3.8G    /home/saml/Downloads/audible/audio_books
3.8G    /home/saml/Downloads/audible
2.3G    /home/saml/Downloads/apps_archive
1.5G    /home/saml/Downloads/digital_blasphemy/db1440ppng.zip
1.5G    /home/saml/Downloads/digital_blasphemy
Don't show me the directory, just the files:
$ du -ah ~/Downloads/ | grep -v "/$" | sort -rh | head -6 
3.8G    /home/saml/Downloads/audible/audio_books
3.8G    /home/saml/Downloads/audible
2.3G    /home/saml/Downloads/apps_archive
1.5G    /home/saml/Downloads/digital_blasphemy/db1440ppng.zip
1.5G    /home/saml/Downloads/digital_blasphemy
835M    /home/saml/Downloads/apps_archive/cad_cam_cae/salome/Salome-V6_5_0-LGPL-x86_64.run
If you just want the list of smallest to biggest, but the top 6 offending files you can reverse the sort switch, drop (-r), and use tail -6 instead of the head -6.
$ du -ah ~/Downloads/ | grep -v "/$" | sort -h | tail -6
835M    /home/saml/Downloads/apps_archive/cad_cam_cae/salome/Salome-V6_5_0-LGPL-x86_64.run
1.5G    /home/saml/Downloads/digital_blasphemy
1.5G    /home/saml/Downloads/digital_blasphemy/db1440ppng.zip
2.3G    /home/saml/Downloads/apps_archive
3.8G    /home/saml/Downloads/audible
3.8G    /home/saml/Downloads/audible/audio_books

Sunday, May 15, 2016

create csv using php

http://php.net/manual/en/function.fputcsv.php


<?php

$list 
= array (
    array(
'aaa''bbb''ccc''dddd'),
    array(
'123''456''789'),
    array(
'"aaa"''"bbb"')
);
$fp fopen('file.csv''w');

foreach (
$list as $fields) {
    
fputcsv($fp$fields);
}
fclose($fp);?>

Saturday, November 14, 2015

PHP: fopen error handling

http://stackoverflow.com/questions/24753821/php-fopen-error-handling

I do fetch a file with
$fp = fopen('uploads/Team/img/'.$team_id.'.png', "rb");
$str = stream_get_contents($fp);
fclose($fp);
and then the method gives it back as image. But when fopen() fails, because the file did not exists, it throws an error:
[{"message":"Warning: fopen(uploads\/Team\/img\/1.png): failed to open stream: No such file or directory in C:\...
This is coming back as json, obviously.
The Question is now: How can i catch the error and prevent the method from throwing this error directly to the client?
shareimprove this question

    
i tried something like this if($fp = fopen('uploads/Team/img/'.$team_id.'.png', "rb")){ throw this->createNotFoundException('No image found for id '.$team_id); } but it didnt worked. – humpdi Jul 15 '14 at 8:54
    
i did also tried try catch block, but didnt worked. the error was readable to the client. – humpdi Jul 15 '14 at 8:56
    
try { $fp = fopen('uploads/Team/img/'.$team_id.'.png', "rb"); } catch(Exception $e) { throw $this->createNotFoundException('No image found for id '.$team_id); } – humpdi Jul 15 '14 at 8:58

3 Answers

up vote 5 down vote accepted
You should first test the existence of a file by file_exists().
    try
    {
      $fileName = 'uploads/Team/img/'.$team_id.'.png';

      if ( !file_exists($fileName) ) {
        throw new Exception('File not found.');
      }

      $fp = fopen($fileName, "rb");
      if ( !$fp ) {
        throw new Exception('File open failed.');
      }  
      $str = stream_get_contents($fp);
      fclose($fp);

      // send success JSON

    } catch ( Exception $e ) {
      // send error message if you can
    } 
or simple solution without exceptions:
    $fileName = 'uploads/Team/img/'.$team_id.'.png';
    if ( file_exists($fileName) && ($fp = fopen($fileName, "rb"))!==false ) {

      $str = stream_get_contents($fp);
      fclose($fp);

      // send success JSON    
    }
    else
    {
      // send error message if you can  
    }
shareimprove this answer

    
thanks a lot dude, thats the point! :) – humpdi Jul 15 '14 at

failed to open stream: No such file or directory

http://stackoverflow.com/questions/16934912/failed-to-open-stream-no-such-file-or-directory

Can anyone help with this one? I am new to web developing and not sure what this error means?
Warning: fopen(images/nophoto.png): failed to open stream: No such file or directory in /home/u835626360/public_html/remove.html on line 101
can't this file/picture is open you need close
CODE:
$expire=time()-3600;
setcookie("dname","a", $expire);
setcookie("dpode","a", $expire);
}
function delpics($filename)
{
$path_to_file='userpics/';
$old = getcwd(); // Save the current directory
    chdir($path_to_file);
    $fh = fopen($filename, 'w') or die("can't this file/picture is open you need close ");
    fclose($fh);
    if (!unlink($filename))
  {
  echo ("Error deleting $file");
  }
else
  {
  echo ("Deleted  $filename");
  }
    chdir($old); // Restore the old working directory   
}
shareimprove this question

    
it said, file not found – Raptor Jun 5 '13 at 8:25
    
Can you at least post the full path to the image file ? – Jerska Jun 5 '13 at 8:28
    
sorry what does this mean? – user2149630 Jun 5 '13 at 11:49

3 Answers

You need to give fopen the full path of the file, and you don't need chdir() at all. Try this version:
$path_to_file='userpics/';
$fh = fopen($path_to_file.$filename, 'w') or die('Permission error');
shareimprove this answer


PHP: fopen: No such file or directory

http://stackoverflow.com/questions/10877007/php-fopen-no-such-file-or-directory

I am trying to create write a log file for my web site. To do this I use the following code to try and open the file. Now the file does not exist yet, but the documentation states that adding "a+" flag ensures that the file is created if it does not exist.
 $file = fopen($_SERVER['DOCUMENT_ROOT']."/logs/mylogfile.txt", "a+");
The above code gives me the following error...
Warning: fopen(E:/wamp/www/logs/mylogfile.txt) [function.fopen]: failed to open stream: No such file or directory
What am I doing wrong ? Please excuse me if this is stupid question, I am very new to PHP.
shareimprove this question

    
Do you have write permissions to that folder? – alexn Jun 4 '12 at 6:13
    
did you checked your E:/wamp/www/logs/ folder for any file named "tagMetroLog.txt" ? – Miqdad Ali Jun 4 '12 at 6:15
    
Thanks for the response.. how can I check that ? – Heshan Perera Jun 4 '12 at 6:17
1  
yes.. the folder named logs doesn't event exists. – Heshan Perera Jun 4 '12 at 6:17
    
Create a folder inside your www folder named logs and create the file "tagMetroLog.txt" or you can use $file = @fopen($_SERVER['DOCUMENT_ROOT']."/logs/mylogfile.txt", "a+"); It will not show any error – Miqdad Ali Jun 4 '12 at 6:22

1 Answer

up vote 8 down vote accepted
fopen's 2nd parameter "a+" can only create the file if the directory exists. Make sure the logs directory is there. If it's not the case use:
mkdir($_SERVER['DOCUMENT_ROOT']."/logs/", 0777, true);
(true is the key) before fopen()

PHP fopen failed to open stream: No such file or directory

https://community.spiceworks.com/topic/268950-php-fopen-failed-to-open-stream-no-such-file-or-directory


re you trying to include a URL path or via Samba/NetBIOS or something else? What OS is it using? Can you do something like:
print_r(scandir ($filepath));
.. and it will display a list of files so you're certain it actually can see the path?
'w' won't create the file if it doesn't exist on open, it just opens it for writing. if the file doesn't exist, it doesn't. I'd do something like:
if (!file_exists($filepath)) {
touch($filepath);
}

failed to open stream: Inappropriate ioctl for device

https://prggmr.wordpress.com/2011/05/12/failed-to-open-stream-inappropriate-ioctl-for-device/


//
you're reading...
PHP

failed to open stream: Inappropriate ioctl for device

A new error I have yet to encounter at first glance this was a bit strange and didn’t make much sense … putting on my System administration hat the answer was quite obvious ownership.

Error:

failed to open stream: Inappropriate ioctl for device

Solution:

chown owner:group file.php
Modify the ownership of the file throwing the error to match that of the file that is being included.
Headache solved!

Basic PHP File Handling — Create, Open, Read, Write, Append, Close, and Delete

https://davidwalsh.name/basic-php-file-handling-create-open-read-write-append-close-delete

I don't do a great deal of file handling in my PHP code -- most of my customers don't have a need for it or there's no room for file creation in the already tight budget. On the rare occasion that I do need to manipulate files, I keep the following tip sheet.

Create a File

$my_file = 'file.txt';
$handle = fopen($my_file, 'w') or die('Cannot open file:  '.$my_file); //implicitly creates file

Open a File

$my_file = 'file.txt';
$handle = fopen($my_file, 'w') or die('Cannot open file:  '.$my_file); //open file for writing ('w','r','a')...

Read a File

$my_file = 'file.txt';
$handle = fopen($my_file, 'r');
$data = fread($handle,filesize($my_file));

Write to a File

$my_file = 'file.txt';
$handle = fopen($my_file, 'w') or die('Cannot open file:  '.$my_file);
$data = 'This is the data';
fwrite($handle, $data);

Append to a File

$my_file = 'file.txt';
$handle = fopen($my_file, 'a') or die('Cannot open file:  '.$my_file);
$data = 'New data line 1';
fwrite($handle, $data);
$new_data = "\n".'New data line 2';
fwrite($handle, $new_data);

Close a File

$my_file = 'file.txt';
$handle = fopen($my_file, 'w') or die('Cannot open file:  '.$my_file);
//write some data here
fclose($handle);

Delete a File

$my_file = 'file.txt';
unlink($my_file);

move_uploaded_file gives “failed to open stream: Permission denied ” error after all configurations i did


http://stackoverflow.com/questions/8103860/move-uploaded-file-gives-failed-to-open-stream-permission-denied-error-after

I keep getting this error when trying to configure the upload directory with Apache 2.2 and PHP 5.3 on CentOS.
In php.ini:
upload_tmp_dir = /var/www/html/mysite/tmp_file_upload/
In httpd.conf:
Directory /var/www/html/mysite/tmp_file_upload/>
    Options  -Indexes
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>
<Directory /var/www/html/mysite/images/>
                Options -Indexes
</Directory>
CentOS directory permissions:
drwxrwxr-x 2 root root 4096 Nov 11 10:01 images
drwxr-xr-x 2 root root 4096 Nov 12 04:54 tmp_file_upload
No matter what I do, I keep getting this error from PHP when I upload the file:
Warning: move_uploaded_file(images/robot.jpg): failed to open stream: Permission denied in /var/www/html/mysite/process.php on line 78
Warning: move_uploaded_file(): Unable to move '/tmp/phpsKD2Qm' to 'images/robot.jpg' in /var/www/html/mysite/process.php on line 78
As you can see it never did take the configuration from the php.ini file regarding the upload file.
What am I doing wrong here?
shareimprove this question

    
775? Maybe your server is running as nobody. Only root can write in this case (your "images" permissions)... – xfix Nov 12 '11 at 10:34
    
what does it means ? how can i change it ? – user63898 Nov 12 '11 at 11:16

7 Answers

up vote 111 down vote accepted
This is because images and tmp_file_upload are only writable by root user. For upload to work we need to make the owner of those folders same as httpd process owner OR make them globally writable (bad practice).
  1. Check apache process owner: $ps aux | grep httpd. The first column will be the owner typically it will be nobody
  2. Change the owner of images and tmp_file_upload to be become nobody or whatever the owner you found in step 1.
    $sudo chown nobody /var/www/html/mysite/images/
    $sudo chown nobody /var/www/html/mysite/tmp_file_upload/
  3. Chmod images and tmp_file_upload now to be writable by the owner, if needed [Seems you already have this in place]. Mentioned in @Dmitry Teplyakov answer.
    $ sudo chmod -R 0755 /var/www/html/mysite/images/
    $ sudo chmod -R 0755 /var/www/html/mysite/tmp_file_upload/
  4. For more details why this behavior happend, check the manual http://php.net/manual/en/ini.core.php#ini.upload-tmp-dir , note that it also talking about open_basedir directive.

Wednesday, October 28, 2015

7 tips to prevent PHP running out of memory

http://v1.srcnix.com/2010/02/10/7-tips-to-prevent-php-running-out-of-memory/

Is the following error familiar to you?
Fatal error: Allowed memory size of XXX bytes exhausted
It is to me and recently I’ve written an import script that reads the contents of 10 XML files. These files accumulate a total of 14.9MB in size. The XML files contain page content (A good 10,000), each page has references to images and PDFs which needed to be downloaded, stored on the filesystem in the same directory structure as well as imported into an asset manager, again in the same structure. Once the assets have been downloaded the script stores the contents of the XML files.
Poor programming could result in the script running out of memory rather quickly, there are a great deal of factors that could cause this – not just improper programming techniques but lack of knowledge in what affects memory. Because of this I’ve decided to write a few notes and tips on improving performance and memory usage.
Memory for PHP is cleared up my PHPs garbage collector. Sadly the garbage collector is a little lazy. In fact, it never seems to get to work on time and as a result memory is not freed quick enough for the scripts to progress. To work inline with the garbage collector here are a few tips on improving your code:

Tip 1 (Knowing what parts of your script is using the most memory)
If you’re looking to find out where your script is running out of memory the following function would be your best bet:
memory_get_peak_usage();
This function will return the current memory usage at the point it is executed. You’ll eventually see where your script is using the most amount of memory, or in my case, where the garbage collector decided it would rather go on lunch than do its job.

Tip 2 (Reassign null to vars along with un-setting them)

The unset(); function is useful when the garbage collector is doing its rounds however until then the unset(); function simply destroys the variable reference to the data, the data still exists in memory and PHP sees the memory as inuse despite no longer having a pointer to it. The solution: Assign null to your variables to clear the data, at least until the garbage collector gets ahold of it.
$var = null;
You can also use unset(); to unset the variable pointer, however there is little difference in memory usage, as far as I can see:
unset($var);

Tip 3 (__destruct your object references upon disposing of an object)

PHP does not release memory dedicated to an objects internal references to other objects until the garbage collector gets round to it. Because of this it’s worth adding a __destruct method to your objects which unsets all references to other objects. This can drastically help lower memory usage and is often ignored.
protected function __distruct()
{
  $this-&gt;childObject = null;
}

Tip 4 (Use functions where possible)

Upon the ending of an in use function PHP clears the memory it was using, at least more efficiently than if not using a function. If you are using recursive code or something similar that is memory intensive try putting the code into a function or method, upon closing of the function/method the memory used for the function will be garbaged much more efficiently than that of unsetting variables within the loop itself.

Tip 5 (Cache your filesystem checks, such as file_exists)

Checking if a file or directory exists before creating it, knowing a directory may be checked more than once? Using file_exists(); costs memory, not much, granted, but it does. The solution? Store the file paths you’ve already checked in an array (Or object property) and use in_array();
if(!in_array($path, $this-&gt;path_list))
{
  // ... Your code to deal with the file
}

Tip 6 (Database access is expensive)

Accessing a database is expensive, if you don’t have to, don’t. Based on tip 5, I am checking the filesystem despite being able to access the database to see if an asset exists. A lot more processing is required to access a database than there is checking the filesystem.

Tip 7 (Do not use objects if not needed)

Unless little or no performance difference I find it best to not using objects when obtaining data from a database, unless updating them. Objects add extra overhead for your scripts, as a result I tend to follow this rule of thumb:
- If not writing to a database, do not use objects, use arrays
- If you will be modifying data in a database, use objects – it tends to be cleaner and somewhat more efficient in the long run.

More tips?

Hopefully these tips will come in use for you. If you happen to have any tips you want to share feel free to post them here.

http://v1.srcnix.com/2010/02/10/7-tips-to-prevent-php-running-out-of-memory/