Class RubyLex::BufferedReader
In: parsers/parse_rb.rb
Parent: Object

Read an input stream character by character. We allow for unlimited ungetting of characters just read.

We simplify the implementation greatly by reading the entire input into a buffer initially, and then simply traversing it using pointers.

We also have to allow for the here document diversion. This little gem comes about when the lexer encounters a here document. At this point we effectively need to split the input stream into two parts: one to read the body of the here document, the other to read the rest of the input line where the here document was initially encountered. For example, we might have

  do_something(<<-A, <<-B)
    stuff
    for
  A
    stuff
    for
  B

When the lexer encounters the <<A, it reads until the end of the line, and keeps it around for later. It then reads the body of the here document. Once complete, it needs to read the rest of the original line, but then skip the here document body.

Methods

Public Class methods

[Source]

     # File parsers/parse_rb.rb, line 346
346:     def initialize(content, options)
347:       @options = options
348: 
349:       if /\t/ =~ content
350:         tab_width = @options.tab_width
351:         content = content.split(/\n/).map do |line|
352:           1 while line.gsub!(/\t+/) { ' ' * (tab_width*$&.length - $`.length % tab_width)}  && $~ #`
353:           line
354:         end .join("\n")
355:       end
356:       @content   = content
357:       @content << "\n" unless @content[-1,1] == "\n"
358:       @size      = @content.size
359:       @offset    = 0
360:       @hwm       = 0
361:       @line_num  = 1
362:       @read_back_offset = 0
363:       @last_newline = 0
364:       @newline_pending = false
365:     end

Public Instance methods

[Source]

     # File parsers/parse_rb.rb, line 367
367:     def column
368:       @offset - @last_newline
369:     end

[Source]

     # File parsers/parse_rb.rb, line 421
421:     def divert_read_from(reserve)
422:       @content[@offset, 0] = reserve
423:       @size      = @content.size
424:     end

[Source]

     # File parsers/parse_rb.rb, line 402
402:     def get_read
403:       res = @content[@read_back_offset...@offset]
404:       @read_back_offset = @offset
405:       res
406:     end

[Source]

     # File parsers/parse_rb.rb, line 371
371:     def getc
372:       return nil if @offset >= @size
373:       ch = @content[@offset, 1]
374: 
375:       @offset += 1
376:       @hwm = @offset if @hwm < @offset
377: 
378:       if @newline_pending
379:         @line_num += 1
380:         @last_newline = @offset - 1
381:         @newline_pending = false
382:       end
383: 
384:       if ch == "\n"
385:         @newline_pending = true
386:       end
387:       ch
388:     end

[Source]

     # File parsers/parse_rb.rb, line 390
390:     def getc_already_read
391:       getc
392:     end

[Source]

     # File parsers/parse_rb.rb, line 408
408:     def peek(at)
409:       pos = @offset + at
410:       if pos >= @size
411:         nil
412:       else
413:         @content[pos, 1]
414:       end
415:     end

[Source]

     # File parsers/parse_rb.rb, line 417
417:     def peek_equal(str)
418:       @content[@offset, str.length] == str
419:     end

[Source]

     # File parsers/parse_rb.rb, line 394
394:     def ungetc(ch)
395:       raise "unget past beginning of file" if @offset <= 0
396:       @offset -= 1
397:       if @content[@offset] == ?\n
398:         @newline_pending = false
399:       end
400:     end

[Validate]